Public Data Access Overview / Discussion Forum

Hello,

I've recently been getting a "HTTP 500 - Server Error" when attempting to access snapshots of Illustris-1-Dark through the API, and the links to individual chunks (e.g. http://www.illustris-project.org/api/Illustris-1-Dark/files/snapshot-135.10.hdf5) do not appear to be working either. Everything was working as of a few days ago. Is this just a temporary server glitch?

Thanks,

-Simon

Okay, it looks like this has been resolved. :)

Cheers,

-Simon

Hi Simon,

You're right this was down for a bit due to some filesystem maintenance, but should be all working again. Let me know if not.

Hi Dylan,

I have also been having another issue, which is different but somewhat related. When I try to use the API to request certain information from the snapshots, sometimes the download does not complete properly, and I end up with an incomplete/corrupt HDF5 file. When this happens, the filesize is much smaller than it should be.

Here's an example:

request_fields = {'gas':'Coordinates,Masses',
                  'dm':'Coordinates',
                  'stars':'Coordinates,Masses',
                  'bhs':'Coordinates,Masses'}
test_data = get('http://www.illustris-project.org/api/Illustris-1/files/snapshot-85.233.hdf5',request_fields,outdir=outdir)

When I have been trying this today, I end up with a ~10MB file (different every time), whereas the desired information should take up ~600MB. Is this some kind of HTTP issue related to the size of the request, or something else? Thanks!

Hi Simon,

You're right, this is an issue related to "cutout" requests above a certain size (~500MB), which I haven't been able to pin down exactly.

The most reasonable suggestion is to split up the request into e.g. 3 separate requests of (gas), (dm), and (stars,bhs).

Hi Dylan,

Okay, that's good to know. When I try the single-particle-type requests you suggest (e.g. just {'gas':'Coordinates,Masses'}), I still sometimes get incomplete output, which suggests that the problem also occurs for requests below 500MB (the example above should be ~200MB). I'll see if I can find a good workaround.