Serving the converted data¶

On-disk file layout¶

Neuroglancer expects all chunks from a scale to be located in the same directory (flat layout). This is problematic when working with large volumes, because filesystems have problems with very large directories. As a result, a deep layout is used by default when saving the chunks to the filesystem:

Default deep layout: key/x-X/y-Y/z-Z.

The default layout is hierarchical, in order to keep the number of directory entries to a reasonable number. This means that you need to serve the data with URL rewriting.
Flat layout: key/x-X_y-Y_z-Z

Chunks are stored in the layout where Neuroglancer will fetch them, so you do not need to configure any URL rewriting. Do not use with very large datasets. This layout can be used by passing the --flat option to the conversion scripts.

nginx¶

A Docker image running a specially-configured nginx web-server is available for serving the converted data: neuroglancer-docker.

The relevant portion of the nginx configuration is reproduced here:

gzip_static always;
# All browsers that are compatible with Neuroglancer support gzip encoding
gunzip      off;

location ~ ^(.*)/([0-9]+-[0-9]+)_([0-9]+-[0-9]+)_([0-9]+-[0-9]+)$ {
    # Chunks are stored in per-axis sub-directories to prevent
    # having too many files in a single directory
    alias $1/$2/$3/$4;
}

location ~ ^(.*):0$ {
    # Microsoft filesystems do not support colons in file names,
    # but they are needed for pre-computed meshes (e.g. 100:0). As
    # :0 is the most common (only?) suffix in use, we look for a
    # file with that suffix stripped.
    try_files $uri $1.json $1 =404;
}

Apache¶

Alternatively, you serve the pre-computed images using Apache, with the following Apache configuration (e.g. put it in a .htaccess file):

# If you get a 403 Forbidden error, try to comment out the Options directives
# below (they may be disallowed by your server's AllowOverride setting).

<IfModule headers_module>
    # Needed to use the data from a Neuroglancer instance served from a
    # different server (see http://enable-cors.org/server_apache.html).
    Header set Access-Control-Allow-Origin "*"
</IfModule>

# Data chunks are stored in sub-directories, in order to avoid having
# directories with millions of entries. Therefore we need to rewrite URLs
# because Neuroglancer expects a flat layout.
Options FollowSymLinks
RewriteEngine On
RewriteRule "^(.*)/([0-9]+-[0-9]+)_([0-9]+-[0-9]+)_([0-9]+-[0-9]+)$" "$1/$2/$3/$4"

# Microsoft filesystems do not support colons in file names, but pre-computed
# meshes use a colon in the URI (e.g. 100:0). As :0 is the most common (only?)
# suffix in use, we will serve a file that has this suffix stripped.
RewriteCond "%{REQUEST_FILENAME}" !-f
RewriteRule "^(.*):0$" "$1"

<IfModule mime_module>
    # Allow serving pre-compressed files, which can save a lot of space for raw
    # chunks, compressed segmentation chunks, and mesh chunks.
    #
    # The AddType directive should in theory be replaced by a "RemoveType .gz"
    # directive, but with that configuration Apache fails to serve the
    # pre-compressed chunks (confirmed with Debian version 2.2.22-13+deb7u6).
    # Fixes welcome.
    Options Multiviews
    AddEncoding x-gzip .gz
    AddType application/octet-stream .gz
</IfModule>

Serving sharded data¶

Content-Encoding¶

Sharded data must be served without any Content-Encoding header <https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Encoding>_.

HTTP Range request¶

Sharded data must be served by a webserver that supports Range header <https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Range>_.

For development uses, python’s bundled SimpleHTTPServer does not support this <https://github.com/python/cpython/issues/86809>_. Recommended alternatives are:

http-server (NodeJS)<https://www.npmjs.com/package/http-server>_
RangeHTTPServer(Python) <https://github.com/danvk/RangeHTTPServer>_

For production uses, most modern static web servers supports range requests. The below is a list of web servers that were tested and works with sharded volumes.

nginx 1.25.3
httpd 2.4.58
caddy 2.7.5

In addition, most object storage also supports range requests without additional configurations.

Enable Access-Control-Allow-Origin header¶

Access-Control-Allow-Origin <https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Origin>_ will need to be enabled if the volume is expected to be accessed cross origin.