Efficiently Accessing all Particles Around Halo CM
Samuel Goldstein
18 Jun '20
I am working on radially averaging halo density profiles of similar mass halos (i.e. 10^13-10^14 solar mass) in TNG-300 and am wondering if there is an efficient way to access particles which are outside of a particular halo. My issue is that I can't compute some of the profiles up to a large enough radius since the particles aren't included in the halo, but I'd like to access all particles within some fixed radius of the halo CM which extends beyond the boundary of some halos. Is there some way to do this?
Dylan Nelson
18 Jun '20
Hi Samuel,
There is no help here from the data: once outside the FoF "scope" of particles, you'll need to do a global search to make sure you obtain all particles within a chosen (large) distance, to make your radial profiles.
The common approach here would be a binning or tree algorithm.
Option 1: For a homemade approach, if you have a maximum search radius of X (e.g. 2 cMpc/h), then you can bin all particles into a cubic grid with cell sizes X (you can "bin" just by dividing a Coordinate by the cell size and rounding down, for instance, in each dimension). Then for a given search center, just locate the cell it falls in, and consider only that cell and its 3^3-1 = 26 neighboring cells. To rapidly locate particles in a given cell, you can pre-sort them by their cell index.
Option 0: if you are considering a small enough number of halos, just brute-force compute the distance from each halo center to all particles in the box, and take those which are within your search radius.
Option 2: use a more flexible tree like scipy KDTree to handle the tree construction and searches for you.
Samuel Goldstein
18 Jun '20
Great, this is very helpful thanks so much! Is it possible to directly access the positions of all particles in the box, or will I need to load particles one halo at a time?
Dylan Nelson
19 Jun '20
Yes the loadSubset() function can load one or more fields, for all particles in the box.
If you are doing mass profiles for example, you can do the calculation "chunked" (e.g. load only 5% of the particles at a time), accumulating the mass in each bin (around each halo) for each chunk, so that the calculation can be done on even a low memory machine.
I am working on radially averaging halo density profiles of similar mass halos (i.e. 10^13-10^14 solar mass) in TNG-300 and am wondering if there is an efficient way to access particles which are outside of a particular halo. My issue is that I can't compute some of the profiles up to a large enough radius since the particles aren't included in the halo, but I'd like to access all particles within some fixed radius of the halo CM which extends beyond the boundary of some halos. Is there some way to do this?
Hi Samuel,
There is no help here from the data: once outside the FoF "scope" of particles, you'll need to do a global search to make sure you obtain all particles within a chosen (large) distance, to make your radial profiles.
The common approach here would be a binning or tree algorithm.
Option 1: For a homemade approach, if you have a maximum search radius of
X(e.g. 2 cMpc/h), then you can bin all particles into a cubic grid with cell sizesX(you can "bin" just by dividing a Coordinate by the cell size and rounding down, for instance, in each dimension). Then for a given search center, just locate the cell it falls in, and consider only that cell and its3^3-1 = 26neighboring cells. To rapidly locate particles in a given cell, you can pre-sort them by their cell index.Option 0: if you are considering a small enough number of halos, just brute-force compute the distance from each halo center to all particles in the box, and take those which are within your search radius.
Option 2: use a more flexible tree like scipy KDTree to handle the tree construction and searches for you.
Great, this is very helpful thanks so much! Is it possible to directly access the positions of all particles in the box, or will I need to load particles one halo at a time?
Yes the
loadSubset()function can load one or more fields, for all particles in the box.If you are doing mass profiles for example, you can do the calculation "chunked" (e.g. load only 5% of the particles at a time), accumulating the mass in each bin (around each halo) for each chunk, so that the calculation can be done on even a low memory machine.