Datasette

An open source multi-tool for exploring and publishing data

Datasette is a tool for exploring and publishing data. It helps people take data of any shape or size and publish that as an interactive, explorable website and accompanying API.

Datasette is aimed at data journalists, museum curators, archivists, local governments and anyone else who has data that they wish to share with the world. It is part of a wider ecosystem of tools and plugins dedicated to making working with structured data as productive as possible.

Explore a demo, watch a presentation about the project or Try Datasette without installing anything using Glitch.

Interested in learning Datasette? Start with the official tutorials.

Support questions, feedback? Join the Datasette Discord.

Contents

• Getting started
  • Play with a live demo
  • Follow a tutorial
  • Datasette in your browser with Datasette Lite
  • Try Datasette without installing anything using Glitch
  • Using Datasette on your own computer
• Installation
  • Basic installation
    • Datasette Desktop for Mac
    • Using Homebrew
    • Using pip
  • Advanced installation options
    • Using pipx
    • Using Docker
  • A note about extensions
• The Datasette Ecosystem
  • sqlite-utils
  • Dogsheep
• CLI reference
  • datasette --help
  • datasette serve
    • datasette --get
    • datasette serve --help-settings
  • datasette plugins
  • datasette install
  • datasette uninstall
  • datasette publish
  • datasette publish cloudrun
  • datasette publish heroku
  • datasette package
  • datasette inspect
  • datasette create-token
• Pages and API endpoints
  • Top-level index
  • Database
  • Table
  • Row
• Publishing data
  • datasette publish
    • Publishing to Google Cloud Run
    • Publishing to Heroku
    • Publishing to Vercel
    • Publishing to Fly
    • Custom metadata and plugins
  • datasette package
• Deploying Datasette
  • Deployment fundamentals
  • Running Datasette using systemd
  • Running Datasette using OpenRC
  • Deploying using buildpacks
  • Running Datasette behind a proxy
    • Nginx proxy configuration
    • Apache proxy configuration
• JSON API
  • Default representation
  • Different shapes
  • Pagination
  • Special JSON arguments
  • Table arguments
    • Column filter arguments
    • Special table arguments
  • Expanding foreign key references
  • Discovering the JSON for a page
  • Enabling CORS
  • The JSON write API
    • Inserting rows
    • Upserting rows
    • Updating a row
    • Deleting a row
    • Creating a table
    • Creating a table from example data
    • Dropping tables
• Running SQL queries
  • Named parameters
  • Views
  • Canned queries
    • Canned query parameters
    • Additional canned query options
    • Writable canned queries
    • Magic parameters
    • JSON API for writable canned queries
  • Pagination
  • Cross-database queries
• Authentication and permissions
  • Actors
    • Using the "root" actor
  • Permissions
    • Defining permissions with "allow" blocks
    • The /-/allow-debug tool
  • Access permissions in metadata
    • Access to an instance
    • Access to specific databases
    • Access to specific tables and views
    • Access to specific canned queries
    • Controlling the ability to execute arbitrary SQL
  • Other permissions in metadata
  • API Tokens
    • datasette create-token
  • Checking permissions in plugins
  • actor_matches_allow()
  • The permissions debug tool
  • The ds_actor cookie
    • Including an expiry time
    • The /-/logout page
  • Built-in permissions
    • view-instance
    • view-database
    • view-database-download
    • view-table
    • view-query
    • insert-row
    • delete-row
    • update-row
    • create-table
    • drop-table
    • execute-sql
    • permissions-debug
    • debug-menu
• Performance and caching
  • Immutable mode
  • Using "datasette inspect"
  • HTTP caching
  • datasette-hashed-urls
• CSV export
  • URL parameters
  • Streaming all records
• Binary data
  • Linking to binary downloads
  • Binary plugins
• Facets
  • Facets in query strings
  • Facets in metadata
  • Suggested facets
  • Speeding up facets with indexes
  • Facet by JSON array
  • Facet by date
• Full-text search
  • The table page and table view API
  • Advanced SQLite search queries
  • Configuring full-text search for a table or view
  • Searches using custom SQL
  • Enabling full-text search for a SQLite table
    • Configuring FTS using sqlite-utils
    • Configuring FTS using csvs-to-sqlite
    • Configuring FTS by hand
  • FTS versions
• SpatiaLite
  • Warning
  • Installation
    • Installing SpatiaLite on OS X
    • Installing SpatiaLite on Linux
  • Spatial indexing latitude/longitude columns
  • Making use of a spatial index
  • Importing shapefiles into SpatiaLite
  • Importing GeoJSON polygons using Shapely
  • Querying polygons using within()
• Metadata
  • Per-database and per-table metadata
  • Source, license and about
  • Column descriptions
  • Specifying units for a column
  • Setting a default sort order
  • Setting a custom page size
  • Setting which columns can be used for sorting
  • Specifying the label column for a table
  • Hiding tables
  • Using YAML for metadata
• Settings
  • Using --setting
  • Configuration directory mode
  • Settings
    • default_allow_sql
    • default_page_size
    • sql_time_limit_ms
    • max_returned_rows
    • max_insert_rows
    • num_sql_threads
    • allow_facet
    • default_facet_size
    • facet_time_limit_ms
    • facet_suggest_time_limit_ms
    • suggest_facets
    • allow_download
    • allow_signed_tokens
    • max_signed_tokens_ttl
    • default_cache_ttl
    • cache_size_kb
    • allow_csv_stream
    • max_csv_mb
    • truncate_cells_html
    • force_https_urls
    • template_debug
    • trace_debug
    • base_url
  • Configuring the secret
  • Using secrets with datasette publish
• Introspection
  • /-/metadata
  • /-/versions
  • /-/plugins
  • /-/settings
  • /-/databases
  • /-/threads
  • /-/actor
  • /-/messages
• Custom pages and templates
  • Custom CSS and JavaScript
    • CSS classes on the <body>
    • Serving static files
    • Publishing static assets
  • Custom templates
  • Custom pages
    • Path parameters for pages
    • Custom headers and status codes
    • Returning 404s
    • Custom redirects
  • Custom error pages
• Plugins
  • Installing plugins
    • One-off plugins using --plugins-dir
    • Deploying plugins using datasette publish
  • Seeing what plugins are installed
  • Plugin configuration
    • Secret configuration values
• Writing plugins
  • Writing one-off plugins
  • Starting an installable plugin using cookiecutter
  • Packaging a plugin
  • Static assets
  • Custom templates
  • Writing plugins that accept configuration
  • Designing URLs for your plugin
  • Building URLs within plugins
• Plugin hooks
  • prepare_connection(conn, database, datasette)
  • prepare_jinja2_environment(env, datasette)
  • extra_template_vars(template, database, table, columns, view_name, request, datasette)
  • extra_css_urls(template, database, table, columns, view_name, request, datasette)
  • extra_js_urls(template, database, table, columns, view_name, request, datasette)
  • extra_body_script(template, database, table, columns, view_name, request, datasette)
  • publish_subcommand(publish)
  • render_cell(row, value, column, table, database, datasette, request)
  • register_output_renderer(datasette)
  • register_routes(datasette)
  • register_commands(cli)
  • register_facet_classes()
  • register_permissions(datasette)
  • asgi_wrapper(datasette)
  • startup(datasette)
  • canned_queries(datasette, database, actor)
  • actor_from_request(datasette, request)
  • filters_from_request(request, database, table, datasette)
  • permission_allowed(datasette, actor, action, resource)
  • register_magic_parameters(datasette)
  • forbidden(datasette, request, message)
  • handle_exception(datasette, request, exception)
  • menu_links(datasette, actor, request)
  • table_actions(datasette, actor, database, table, request)
  • database_actions(datasette, actor, database, request)
  • skip_csrf(datasette, scope)
  • get_metadata(datasette, key, database, table)
• Testing plugins
  • Setting up a Datasette test instance
  • Using pdb for errors thrown inside Datasette
  • Using pytest fixtures
  • Testing outbound HTTP calls with pytest-httpx
  • Registering a plugin for the duration of a test
• Internals for plugins
  • Request object
  • The MultiParams class
  • Response class
    • Returning a response with .asgi_send(send)
    • Setting cookies with response.set_cookie()
  • Datasette class
    • .databases
    • .permissions
    • .plugin_config(plugin_name, database=None, table=None)
    • await .render_template(template, context=None, request=None)
    • await .permission_allowed(actor, action, resource=None, default=...)
    • await .ensure_permissions(actor, permissions)
    • await .check_visibility(actor, action=None, resource=None, permissions=None)
    • .create_token(actor_id, expires_after=None, restrict_all=None, restrict_database=None, restrict_resource=None)
    • .get_database(name)
    • .add_database(db, name=None, route=None)
    • .add_memory_database(name)
    • .remove_database(name)
    • .sign(value, namespace="default")
    • .unsign(value, namespace="default")
    • .add_message(request, message, type=datasette.INFO)
    • .absolute_url(request, path)
    • .setting(key)
    • .resolve_database(request)
    • .resolve_table(request)
    • .resolve_row(request)
    • datasette.client
    • datasette.urls
  • Database class
    • Database(ds, path=None, is_mutable=True, is_memory=False, memory_name=None)
    • db.hash
    • await db.execute(sql, ...)
    • Results
    • await db.execute_fn(fn)
    • await db.execute_write(sql, params=None, block=True)
    • await db.execute_write_script(sql, block=True)
    • await db.execute_write_many(sql, params_seq, block=True)
    • await db.execute_write_fn(fn, block=True)
    • db.close()
    • Database introspection
  • CSRF protection
  • The _internal database
  • The datasette.utils module
    • parse_metadata(content)
    • await_me_maybe(value)
    • Tilde encoding
  • datasette.tracer
    • Tracing child tasks
  • Import shortcuts
• Contributing
  • General guidelines
  • Setting up a development environment
  • Running the tests
  • Using fixtures
  • Debugging
  • Code formatting
    • Running Black
    • blacken-docs
    • Prettier
  • Editing and building the documentation
    • Running Cog
  • Continuously deployed demo instances
  • Release process
  • Alpha and beta releases
  • Releasing bug fixes from a branch
  • Upgrading CodeMirror
• Changelog
  • 1.0a4 (2023-08-21)
  • 1.0a3 (2023-08-09)
    • Smaller changes
  • 0.64.2 (2023-03-08)
  • 0.64.1 (2023-01-11)
  • 0.64 (2023-01-09)
  • 0.63.3 (2022-12-17)
  • 1.0a2 (2022-12-14)
  • 1.0a1 (2022-12-01)
  • 1.0a0 (2022-11-29)
    • Signed API tokens
    • Write API
  • 0.63.2 (2022-11-18)
  • 0.63.1 (2022-11-10)
  • 0.63 (2022-10-27)
    • Features
    • Plugin hooks and internals
    • Documentation
  • 0.62 (2022-08-14)
    • Features
    • Plugin hooks
    • Bug fixes
    • Documentation
  • 0.61.1 (2022-03-23)
  • 0.61 (2022-03-23)
  • 0.60.2 (2022-02-07)
  • 0.60.1 (2022-01-20)
  • 0.60 (2022-01-13)
    • Plugins and internals
    • Faceting
    • Other small fixes
  • 0.59.4 (2021-11-29)
  • 0.59.3 (2021-11-20)
  • 0.59.2 (2021-11-13)
  • 0.59.1 (2021-10-24)
  • 0.59 (2021-10-14)
  • 0.58.1 (2021-07-16)
  • 0.58 (2021-07-14)
  • 0.57.1 (2021-06-08)
  • 0.57 (2021-06-05)
    • New features
    • Bug fixes and other improvements
  • 0.56.1 (2021-06-05)
  • 0.56 (2021-03-28)
  • 0.55 (2021-02-18)
  • 0.54.1 (2021-02-02)
  • 0.54 (2021-01-25)
    • The _internal database
    • Named in-memory database support
    • JavaScript modules
    • Code formatting with Black and Prettier
    • Other changes
  • 0.53 (2020-12-10)
  • 0.52.5 (2020-12-09)
  • 0.52.4 (2020-12-05)
  • 0.52.3 (2020-12-03)
  • 0.52.2 (2020-12-02)
  • 0.52.1 (2020-11-29)
  • 0.52 (2020-11-28)
  • 0.51.1 (2020-10-31)
  • 0.51 (2020-10-31)
    • New visual design
    • Plugins can now add links within Datasette
    • Binary data
    • URL building
    • Running Datasette behind a proxy
    • Smaller changes
  • 0.50.2 (2020-10-09)
  • 0.50.1 (2020-10-09)
  • 0.50 (2020-10-09)
  • 0.49.1 (2020-09-15)
  • 0.49 (2020-09-14)
  • 0.48 (2020-08-16)
  • 0.47.3 (2020-08-15)
  • 0.47.2 (2020-08-12)
  • 0.47.1 (2020-08-11)
  • 0.47 (2020-08-11)
  • 0.46 (2020-08-09)
  • 0.45 (2020-07-01)
    • Magic parameters for canned queries
    • Log out
    • Better plugin documentation
    • New plugin hooks
    • Smaller changes
  • 0.44 (2020-06-11)
    • Authentication
    • Permissions
    • Writable canned queries
    • Flash messages
    • Signed values and secrets
    • CSRF protection
    • Cookie methods
    • register_routes() plugin hooks
    • Smaller changes
    • The road to Datasette 1.0
  • 0.43 (2020-05-28)
  • 0.42 (2020-05-08)
  • 0.41 (2020-05-06)
  • 0.40 (2020-04-21)
  • 0.39 (2020-03-24)
  • 0.38 (2020-03-08)
  • 0.37.1 (2020-03-02)
  • 0.37 (2020-02-25)
  • 0.36 (2020-02-21)
  • 0.35 (2020-02-04)
  • 0.34 (2020-01-29)
  • 0.33 (2019-12-22)
  • 0.32 (2019-11-14)
  • 0.31.2 (2019-11-13)
  • 0.31.1 (2019-11-12)
  • 0.31 (2019-11-11)
  • 0.30.2 (2019-11-02)
  • 0.30.1 (2019-10-30)
  • 0.30 (2019-10-18)
  • 0.29.3 (2019-09-02)
  • 0.29.2 (2019-07-13)
  • 0.29.1 (2019-07-11)
  • 0.29 (2019-07-07)
    • ASGI
    • New plugin hook: asgi_wrapper
    • New plugin hook: extra_template_vars
    • Secret plugin configuration options
    • Facet by date
    • Easier custom templates for table rows
    • ?_through= for joins through many-to-many tables
    • Small changes
  • 0.28 (2019-05-19)
    • Supporting databases that change
    • Faceting improvements, and faceting plugins
    • datasette publish cloudrun
    • register_output_renderer plugins
    • Medium changes
    • Small changes
  • 0.27.1 (2019-05-09)
  • 0.27 (2019-01-31)
  • 0.26.1 (2019-01-10)
  • 0.26 (2019-01-02)
  • 0.25.2 (2018-12-16)
  • 0.25.1 (2018-11-04)
  • 0.25 (2018-09-19)
  • 0.24 (2018-07-23)
  • 0.23.2 (2018-07-07)
  • 0.23.1 (2018-06-21)
  • 0.23 (2018-06-18)
    • CSV export
    • Foreign key expansions
    • New configuration settings
    • Control HTTP caching with ?_ttl=
    • Improved support for SpatiaLite
    • latest.datasette.io
    • Miscellaneous
  • 0.22.1 (2018-05-23)
  • 0.22 (2018-05-20)
  • 0.21 (2018-05-05)
  • 0.20 (2018-04-20)
  • 0.19 (2018-04-16)
  • 0.18 (2018-04-14)
  • 0.17 (2018-04-13)
  • 0.16 (2018-04-13)
  • 0.15 (2018-04-09)
  • 0.14 (2017-12-09)
  • 0.13 (2017-11-24)
  • 0.12 (2017-11-16)
  • 0.11 (2017-11-14)
  • 0.10 (2017-11-14)
  • 0.9 (2017-11-13)
  • 0.8 (2017-11-13)