584 views
owned this note
# HIP Winter School 2021
This is the learner padd for our winter school. In the following, we will provide helpful information, important questions, directions and feedback. Feel free to contribute.
## Workshop details
The workshop is organized based on this indico webpage: https://events.hifis.net/event/200/overview
### Agenda
#### Nov 25, 2021: HPC introduction
speakers: Lokamani (Scientific Computing/HZDR), David Pape (Scientific Computing/HZDR), Steve Schmerler (Helmholtz AI/HZDR), Peter Steinbach (Helmholtz AI/HZDR)
start: 10am
end: 4/5 pm
#### Dec 2, 2021: introduction to GPU programming
speakers: Guido Juckeland (Scientific Computing/HZDR)
#### Dec 9, 2021: Introduction to pytorch
speakers: Steve Schmerler (Helmholtz AI/HZDR), Peter Steinbach (Helmholtz AI/HZDR)
# Infrastructure / Cluster login
**[General cluster documentation here][horeka_docs]**
**[Jupyter docs](https://www.nhr.kit.edu/userdocs/jupyter)**
## Getting an Account
* We have registered you with the colleagues at KIT for access.
* You should have gotten a first mail from the colleagues in Karlsruhe with "Zugang DESY "Helmholtz Imaging Winter School 2021" as title
* this mail contains instructions how to login
* if you did not get a mail, please get in touch with us and we will give you the email address of our contact at KIT
* You'll receive an email, maybe in German: (user account name changed)
```
Benutzerkennung: ab1234
E-Mail-Adresse: [1]ab1234@partner.kit.edu
Nun müssen Sie sich bei [2]https://fels.scc.kit.edu bei der
Heimatorganisation KIT ein Dienstpasswort für den Horeka setzen und
anschließend benutzen. Zudem müssen sie eine 2-Faktor-Authorisierung
verwenden. Weitere Informationen finden Sie hier
[3]https://www.nhr.kit.edu/userdocs/horeka/login/ .
Informationen zum PW schicke ich getrennt zu.
```
* Get another email with a "password definition":
```
Hallo,
ich schicke Ihnen die Definition des Passwortes in Bezug auf ihre
Benutzerkennung :
-> die ersten beiden Zahl; die zwei Buchstaben (1. Groß- und 2.
Kleinschreibung); die letzten beiden Zahlen; +- (also plus und minus)
```
It means that the password is *derived from the login name*. Using
the example login `ab1234`, this would be: `12Ab34+-`
Use <https://my.scc.kit.edu/shib/passwort.php> to change password later.
* Login to <https://fels.scc.kit.edu> using those credentials. Important:
choose KIT as your organization if your assigned email is
`<login>@partner.kit.edu` because you are treated as an "external partner"
(details in [horeka_docs]).
* Set up 2-factor-authentication (2FA): activate token. Use for example
FreeOTP+ on a phone. It's available from the F-Droid store.
* Register access to the HoreKa machine: set a "service password" (German:
Dienstpasswort). This can be longer than 8 digits and also doesn't have a
fixed pattern.
* Upload an ssh public key and connect it to HoreKa:
* Index > My SSH Pubkeys -> add key; then Home > HoreKa > Set SSH Key --> choose key
## using `ssh` to enter Horeka
* ssh into the machine
* The [docs][horeka_docs_network] say: Even though you have set up an ssh key,
*you need to have an IP from your center (e.g. HZDR)*, so start VPN. Make sure the VPN does not only tunnel traffic to your center, e.g. use check if there is a `nosplit` VPN
* `ssh ab1234@horeka.scc.kit.edu`
* Enter one-time-password (OTP) using previously activated token and
FreeOTP+ app (in this example).
* Enter "service password".
* Good news: another login: no OTP, no "service password". Things seem to be
cached (probably bound to your IP).
## Jupyter Lab
### Software setup
ssh into horeka, then in your home directory, do this
```sh
$ ssh user42@horeka.scc.kit.edu
# on horeka
# important, run this first
$ module load jupyter/tensorflow
# install in your $HOME directory
$ pip install --user --no-deps torchvision
$ pip install --user torchinfo
```
Optional for last part of the afternoon, not required, you can also just watch the live coding of that part:
```sh
$ pip install --user pytorch-lightning
```
### Download scripts
We have prepared scripts with Python code that we will use in the course. You will need to copy-paste the code from those into notebooks during the course. Please download the scripts to your local machine
```shell
$ scp -r user42@horeka.scc.kit.edu:/hkfs/work/workspace/scratch/se6669-haicore-project-ws-hip-2021/scripts .
```
where `user42` is your user name on Horeka. This will create a directory called `scripts`.
### Login, start notebook session
* jupyter login page: https://haicore-jupyter.scc.kit.edu/
* Login with your account for the access to HoreKa (same way as done on https://fels.scc.kit.edu)
* If you received a KIT account please select KIT
* Otherwise select your Home-Institution
* choose "Access HAICORE@KIT resources"
* enter OTP
* click "Enter JupyterHub"
* click "advanced settings"
* select "HIWS" as partition
* select 1..4 GPUs
* select `jupyter/tensorflow`
* click "Spawn"
[horeka_docs]: https://www.nhr.kit.edu/userdocs/horeka
[horeka_docs_network]: https://www.nhr.kit.edu/userdocs/horeka/login/#network-access-limitations