# HIP Winter School 2021 This is the learner padd for our winter school. In the following, we will provide helpful information, important questions, directions and feedback. Feel free to contribute. ## Workshop details The workshop is organized based on this indico webpage: https://events.hifis.net/event/200/overview ### Agenda #### Nov 25, 2021: HPC introduction speakers: Lokamani (Scientific Computing/HZDR), David Pape (Scientific Computing/HZDR), Steve Schmerler (Helmholtz AI/HZDR), Peter Steinbach (Helmholtz AI/HZDR) start: 10am end: 4/5 pm #### Dec 2, 2021: introduction to GPU programming speakers: Guido Juckeland (Scientific Computing/HZDR) #### Dec 9, 2021: Introduction to pytorch speakers: Steve Schmerler (Helmholtz AI/HZDR), Peter Steinbach (Helmholtz AI/HZDR) # Infrastructure / Cluster login **[General cluster documentation here][horeka_docs]** **[Jupyter docs](https://www.nhr.kit.edu/userdocs/jupyter)** ## Getting an Account * We have registered you with the colleagues at KIT for access. * You should have gotten a first mail from the colleagues in Karlsruhe with "Zugang DESY "Helmholtz Imaging Winter School 2021" as title * this mail contains instructions how to login * if you did not get a mail, please get in touch with us and we will give you the email address of our contact at KIT * You'll receive an email, maybe in German: (user account name changed) ``` Benutzerkennung: ab1234 E-Mail-Adresse: [1]ab1234@partner.kit.edu Nun müssen Sie sich bei [2]https://fels.scc.kit.edu bei der Heimatorganisation KIT ein Dienstpasswort für den Horeka setzen und anschließend benutzen. Zudem müssen sie eine 2-Faktor-Authorisierung verwenden. Weitere Informationen finden Sie hier [3]https://www.nhr.kit.edu/userdocs/horeka/login/ . Informationen zum PW schicke ich getrennt zu. ``` * Get another email with a "password definition": ``` Hallo, ich schicke Ihnen die Definition des Passwortes in Bezug auf ihre Benutzerkennung : -> die ersten beiden Zahl; die zwei Buchstaben (1. Groß- und 2. Kleinschreibung); die letzten beiden Zahlen; +- (also plus und minus) ``` It means that the password is *derived from the login name*. Using the example login `ab1234`, this would be: `12Ab34+-` Use <https://my.scc.kit.edu/shib/passwort.php> to change password later. * Login to <https://fels.scc.kit.edu> using those credentials. Important: choose KIT as your organization if your assigned email is `<login>@partner.kit.edu` because you are treated as an "external partner" (details in [horeka_docs]). * Set up 2-factor-authentication (2FA): activate token. Use for example FreeOTP+ on a phone. It's available from the F-Droid store. * Register access to the HoreKa machine: set a "service password" (German: Dienstpasswort). This can be longer than 8 digits and also doesn't have a fixed pattern. * Upload an ssh public key and connect it to HoreKa: * Index > My SSH Pubkeys -> add key; then Home > HoreKa > Set SSH Key --> choose key ## using `ssh` to enter Horeka * ssh into the machine * The [docs][horeka_docs_network] say: Even though you have set up an ssh key, *you need to have an IP from your center (e.g. HZDR)*, so start VPN. Make sure the VPN does not only tunnel traffic to your center, e.g. use check if there is a `nosplit` VPN * `ssh ab1234@horeka.scc.kit.edu` * Enter one-time-password (OTP) using previously activated token and FreeOTP+ app (in this example). * Enter "service password". * Good news: another login: no OTP, no "service password". Things seem to be cached (probably bound to your IP). ## Jupyter Lab ### Software setup ssh into horeka, then in your home directory, do this ```sh $ ssh user42@horeka.scc.kit.edu # on horeka # important, run this first $ module load jupyter/tensorflow # install in your $HOME directory $ pip install --user --no-deps torchvision $ pip install --user torchinfo ``` Optional for last part of the afternoon, not required, you can also just watch the live coding of that part: ```sh $ pip install --user pytorch-lightning ``` ### Download scripts We have prepared scripts with Python code that we will use in the course. You will need to copy-paste the code from those into notebooks during the course. Please download the scripts to your local machine ```shell $ scp -r user42@horeka.scc.kit.edu:/hkfs/work/workspace/scratch/se6669-haicore-project-ws-hip-2021/scripts . ``` where `user42` is your user name on Horeka. This will create a directory called `scripts`. ### Login, start notebook session * jupyter login page: https://haicore-jupyter.scc.kit.edu/ * Login with your account for the access to HoreKa (same way as done on https://fels.scc.kit.edu) * If you received a KIT account please select KIT * Otherwise select your Home-Institution * choose "Access HAICORE@KIT resources" * enter OTP * click "Enter JupyterHub" * click "advanced settings" * select "HIWS" as partition * select 1..4 GPUs * select `jupyter/tensorflow` * click "Spawn" [horeka_docs]: https://www.nhr.kit.edu/userdocs/horeka [horeka_docs_network]: https://www.nhr.kit.edu/userdocs/horeka/login/#network-access-limitations