Get started in the cloud environment

Our alpha users have access to the OPU Zeus through our machine k1 at OVH.

Note

You might have been told to use our other OPU Vulcain. Steps are the same, but replace k1 by k2.

K1 Guide

Create SSH key

SSH key are used to authenticate your computer against the machine hosting the OPU. If you already have one that is protected with a passphrase, you can skip this step.

  • Open Terminal (or Git Bash, if on Windows)
  • Create SSH key: ssh-keygen -t rsa -b 4096
  • When you’re prompted to “Enter a file in which to save the key,” press Enter. This accepts the default file location.
  • At the prompt, type a secure passphrase (Don’t leave the passphrase empty!).
  • Get your SSH public key: copy-paste the content of ~/.ssh/id_rsa.pub with less ~/.ssh/id_rsa.pub
  • Send to support@lighton.io your SSH public key (never communicate your private key!) for account creation.

See also this GitHub tutorial.

If the SSH key is refused

On some systems the SSH key needs to be activated every time you launch a new terminal. It’s not the case on Ubuntu.

  • Start ssh-agent: ssh-agent /bin/bash
  • Add SSH key to ssh-agent: ssh-add ~/.ssh/id_rsa
  • Enter the passphrase of your key

K1 connection

Connect to k1 with: ssh -A username@k1.lighton.io. The -A option allows to forward your SSH identity from your local machine to k1, so that you don’t have to register it again on other remotes.

Replace “username” by the name of the account LightOn gave you on k1.

Once you login, you will be asked a new password.

Note

On Windows machine, don’t use PuTTY, but Git Bash (unless you used PuTTY to create your SSH key pair).

Python virtualenv

A Python 3.5 virtualenv has been created for you and is active. It is recommended to use it as it already contains LightOn’s libraries lightonopu and lightonml. You can add other packages to the virtual environment with the command pip install name-of-module. In case you want to make your own virtualenv, go to Virtual Environments.

Important

You might need to update lightonml or lightonopu libraries. You can do it with the following command: pip install -f /usr/local/lighton/wheels lightonml -U and pip install -f /usr/local/lighton/wheels lightonopu -U

Jupyter server

A Jupyter server is available at https://k1.lighton.io:8888 on your browser. You can login with your k1 username and password.

When you create a Jupyter Notebook, in New, you need to pick Python 3 (LightOn OPU) to use opu-venv virtual environment. In case you have picked Python 3, you need to switch your kernel in Kernel, Change kernel, Python 3 (LightOn OPU) in your notebook.

Available datasets

Many datasets are already available on k1, you can find them in /data1/mldata.

List of datasets that can be loaded with the lightonml.datasets module:

  • MNIST
  • FashionMNIST
  • SignMNIST
  • CIFAR10
  • CIFAR100
  • STL10

ImageNet, PASCAL-VOC2012 and LSUN are inside the same directory, but we don’t offer any loader for them in lightonml. You can use ones from other machine learning framework, such as PyTorch, or write your own data loader.

We can add other datasets on demand.

Using the OPU

Important

Before using the OPU, you need to book a time slot on the web page https://www.supersaas.fr/schedule/lighton/Cloud_bookings. Please note that the field username must be filled with the username used to connect to k1. You won’t be able to connect to the OPU if you don’t fill correctly this field.

Testing the OPU

This is a self-contained script that is runnable on the OPU to test that everything is installed correctly:

from lightonopu.opu import OPU
from lightonml.random_projections.opu import OPURandomMapping
from lightonml.datasets import MNIST
from lightonml.encoding.base import BinaryThresholdEncoder

# load data
(_, _), (X, _) = MNIST()
# encode data
encoder = BinaryThresholdEncoder()
X_encoded = encoder.transform(X)
# initialize opu objects
opu = OPU()
mapping = OPURandomMapping(opu, n_components=10000)
# get random features
random_features = mapping.fit_transform(X_encoded)
print("Successfully projected {} samples using the OPU.".format(len(X)))

Attention

Do not forget to free the OPU at the end of your booking by doing Close and Halt on your Jupyter Notebook.