This interface is an experiment in the exploration, visualization, and dissemination of massive data sets -- in particular, those generated by large, astrophysical simulations such as Illustris. With total data volume of the order ~0.5 PB, all three of these tasks become increasingly difficult. A powerful approach is thin-client interaction with derived data products (e.g. group catalogs, merger trees) with a relational database, combined with on-demand, server side data processing of the raw data (e.g. full snapshots). Leveraging many modern web technologies we can build this interface within the browser, separating the user from the details of the data storage and processing, and making access instant and platform agnostic.
A full box slice of the simulation is shown in projection, with a depth of 15 Mpc/h, revealing a fifth of the total volume of Illustris at z=0. All the imagery is pre-computed and saved as hierarchical image pyramids. Fast data search is handled by a RDBMS, structured around the group catalogs. The results allow for near instant data extraction from the entire dataset, which can then be visualized within the browser or returned in native (HDF5) format.
The image layers are 2^17 pixels on a side, for a total of ~16 Gigapixels each. Gas projections are ray traced through the volume with an adaptive stepsize, sampling the N-nearest neighbors along each ray segment according to an inverse distance weighting type scheme. X-ray luminosity is proportional to n^2 * sqrt(T). The SZ y-parameter is proportional to n_e * T. Dark matter annihilation is proportional to rho_DM^2. The DM density layer splats each dark matter particle with a SPH-type kernel with radius scaling with the local DM point density. The stellar composite layer splats individual star particles with a Gaussian kernel, with FWHM scaling with the local baryon particle density. Stellar luminosities are calculated in several Johnson/SDSS filters following BC03, and the RGB composite is formed from the (r,g,B) bands.
The site makes use of many exceptional pieces of software, including: MySQL, Django (Python Web Framework), Leaflet (Tile Map), Bootstrap (HTML/CSS Framework), jQuery (JavaScript Framework), three.js (WebGL), d3.js (JS Data Visualization). Smaller components include: leaflet-sidebar, leaflet-groupedlayers, GeoDjango, spinner.js, OculusBridge, chroma.js.
The 'Explorer' frontend interface, backend functionality, and gas image layers were created by Dylan Nelson.
The DM density layer is courtesy of Mark Vogelsberger. The stellar image layer is courtesy of Shy Genel. The individual stellar images are courtesy of Paul Torrey.
Questions/comments? Contact Dylan Nelson.