I am trying to study dark matter defficient galaxies using the TNG100-1 simulation. In order to do that, I chose a mass range and analyzed all the subhalos that were found in it. I am now working with JupyterLab and trying to do the same with their hosts, but I am having some trouble when I form their corresponding URLs.
The main problem is that sometimes, when the code starts analyzing the hosts' URL, it somehow reaches a forbidden one, so I get a message that says that the URL I am asking for is forbidden. I also get other error messages regarding the links, such as
However, these errors do not occur with all of the hosts: the code starts running and it starts forming the URLs but at some point this happens, so it can not keep on going from there on, having previously worked for other hosts. Moreover, this does not always happen to the same host (I traced their IDs so to know which is the one I am analyzing), sometimes it works properly and sometimes it does not for the same host. I thought about excluding the problematic halos from my loop but, because it is not always the same halo that has a forbidden or a broken URL, I cannot do that.
I wanted to know if there is any solution for this problem or any suggestion you can make me for me to run the code without having it interrupted by these errors.
Kind regards,
Elisa Tau.
Dylan Nelson
16 Sep '20
Hi Elisa,
If it only happens sometimes, and if you can repeat for the same exact URL and sometime it works, then this sounds just like a busy server. I will look into it more.
But, practically, two things:
(1) You can simply check the return code, if it is e.g. 502 or 503, you can wait a few seconds, then retry. In this way, eventually all hosts will succeed.
(2) More importantly, if you're using the JupyterLab, you never need to use URLs into the API! You should be using the load functions as described on the Examples Scripts page. If it isn't clear how, please post more exactly what you're doing.
Elisa Tau
16 Sep '20
Hi Dylan,
I have been retrying to run the code once it stops due to these errors but, as it takes lots of hours for it to finish running, this is not the best option. Sometimes it runs for 9 hours and then it shows those return codes, so I have to restart the process for it to stop again a couple of hours later.
I did not know about the load functions you mentioned. I will look into it and try to use them, maybe that will help. If I cannot succeed in applying the method you suggested I will write to you again.
Thank you for your help!
Elisa Tau
23 Sep '20
Hello again,
I still cannot run my code due to the server errors I mentioned before, so I wanted to ask you some more questions.
First of all, I wanted to know if there is a way of skipping the problematic subhalo and continuing with the following one, in order not to stop the code from running when the server is having problems.
Secondly, I wanted to know how can I use and implement on JupyterLab the functions you mentioned. I want to extract the gas, dark matter, stellar and black holes mass of the subhalos within a range of mass and their positions (x,y,z) too. I also want to extract some information regarding their hosts, such as the host's position and its R200. How could I do this by using the functions?
Best regards,
Elisa.
Dylan Nelson
24 Sep '20
Hi Elisa,
Once you are logged in to the Lab, you will find a notebook called "tutorial" in the examples/ directory. This walks through everything you need, following the documentation link I put above.
Elisa Tau
27 Sep '20
I found the tutorial and I will start following the instructions. Thank you for your help, Dylan!
Hello!
I am trying to study dark matter defficient galaxies using the TNG100-1 simulation. In order to do that, I chose a mass range and analyzed all the subhalos that were found in it. I am now working with JupyterLab and trying to do the same with their hosts, but I am having some trouble when I form their corresponding URLs.
The main problem is that sometimes, when the code starts analyzing the hosts' URL, it somehow reaches a forbidden one, so I get a message that says that the URL I am asking for is forbidden. I also get other error messages regarding the links, such as
However, these errors do not occur with all of the hosts: the code starts running and it starts forming the URLs but at some point this happens, so it can not keep on going from there on, having previously worked for other hosts. Moreover, this does not always happen to the same host (I traced their IDs so to know which is the one I am analyzing), sometimes it works properly and sometimes it does not for the same host. I thought about excluding the problematic halos from my loop but, because it is not always the same halo that has a forbidden or a broken URL, I cannot do that.
I wanted to know if there is any solution for this problem or any suggestion you can make me for me to run the code without having it interrupted by these errors.
Kind regards,
Elisa Tau.
Hi Elisa,
If it only happens sometimes, and if you can repeat for the same exact URL and sometime it works, then this sounds just like a busy server. I will look into it more.
But, practically, two things:
(1) You can simply check the return code, if it is e.g. 502 or 503, you can wait a few seconds, then retry. In this way, eventually all hosts will succeed.
(2) More importantly, if you're using the JupyterLab, you never need to use URLs into the API! You should be using the load functions as described on the Examples Scripts page. If it isn't clear how, please post more exactly what you're doing.
Hi Dylan,
I have been retrying to run the code once it stops due to these errors but, as it takes lots of hours for it to finish running, this is not the best option. Sometimes it runs for 9 hours and then it shows those return codes, so I have to restart the process for it to stop again a couple of hours later.
I did not know about the load functions you mentioned. I will look into it and try to use them, maybe that will help. If I cannot succeed in applying the method you suggested I will write to you again.
Thank you for your help!
Hello again,
I still cannot run my code due to the server errors I mentioned before, so I wanted to ask you some more questions.
First of all, I wanted to know if there is a way of skipping the problematic subhalo and continuing with the following one, in order not to stop the code from running when the server is having problems.
Secondly, I wanted to know how can I use and implement on JupyterLab the functions you mentioned. I want to extract the gas, dark matter, stellar and black holes mass of the subhalos within a range of mass and their positions (x,y,z) too. I also want to extract some information regarding their hosts, such as the host's position and its R200. How could I do this by using the functions?
Best regards,
Elisa.
Hi Elisa,
Once you are logged in to the Lab, you will find a notebook called "tutorial" in the
examples/directory. This walks through everything you need, following the documentation link I put above.I found the tutorial and I will start following the instructions. Thank you for your help, Dylan!