Differences

This shows you the differences between two versions of the page.

--- working_with_python [2023/11/03 15:42] – [Jupyter Notebooks] jansen
+++ working_with_python [2025/04/07 15:07] (current) – Modified example for RHEL9 jansen
@@ Line 73: / Line 73: @@
 === Method 1 - subsection Incompatible versions ===
-Unfortunately, python's 'user' directory is independent of the operating system version, but most of the compute nodes including the LOFAR cluster, run RedHat Enterprise Linux, which is sufficiently different to cause packages installed on the desktop not to work all the time.
+Unfortunately, python's 'user' directory is independent of the operating system version, but most of the compute nodes including VDESK, LOFAR cluster and other compute nodes, run RedHat Enterprise Linux, which is sufficiently different to cause packages installed on the desktop not to work all the time.
 In cases like this, it might be necessary to create a separate python user directory structure for those machines:
@@ Line 79: / Line 79: @@
 Add to your .bashrc something like this:
   if [ ! -f /etc/fedora-release ]; then
-    export PYTHONUSERBASE=$HOME/.local-rhel7
+    export PYTHONUSERBASE=$HOME/.local-rhel9
   fi
 For users of the ''tcsh'' shell, add this to your .tcshrc in stead:
   if (! -f /etc/fedora-release) then
-    setenv PYTHONUSERBASE $HOME/.local-rhel7
+    setenv PYTHONUSERBASE $HOME/.local-rhel9
   endif
-And make sure to create that directory ~/.local-rhel7. Now the pip --user commands on RHEL7 machines will install into that newly created directory in stead of the default one used by the desktop systems.
+And make sure to create that directory ~/.local-rhel9. Now the pip --user commands on RHEL9 machines will install into that newly created directory in stead of the default one used by the desktop systems.
-==== METHOD 2: virtualenv ====
+==== METHOD 2: venv ====
+''venv'' is a tool that creates isolated Python environments; it replaces the obsolete ''virtualenv'' that provided similar functionality for python 2.x. A python environment is essentially  a folder which contains copies of all necessary files needed for a Python project to run. In addition each virtual environment will contain a copy of the utility pip to manage packages. For example, let us suppose you would like to install ''pymatlab'' which is not installed on the departmental workstations, then you could do
+<code bash>
+$ mkdir /data2/username/venvs
+$ python3 -m venv /data2/username/venvs/pymatlab
+</code>
+to create a virtual environment (folder) called pymatlab (note that this example explicitly creates this in a directory on your local ''/data2'' disk, in order to avoid running out of disk quota in your home directory, which can easily happen since venvs can become rather big).
+In the example, we use ''python3'' as the python for this environment; if there are multiple python versions on the system, and you want to base your venv on a specific version, use that version to create the venv, e.g. ''python3.12 -m venv /data2/username/venvs/pymatlab''.
+The last step before starting to use the newly generated environment is to activate it, that is to prepend its ''/bin'' folder to your $PATH environment variable. This is done by issuing
+<code bash>
+source /data2/username/pymatlab/bin/activate # bash shells
+source /data2/username/pymatlab/bin/activate.csh # c shells
+</code>
+To acknowledge the activation of pymatlab, the terminal prompt will be changed to
+<code bash>
+(pymatlab)username@hostname:~/python_virt_envs/pymatlab$
+</code>
+to emphasize that you  are operating in a virtual environment. To install pymatlab (or any other package) locally (in your virtual environment) run pip within that environment
+<code bash>
+pip install  pymatlab
+</code>
+Your virtual environment now should have the same core python packages defined globally for all the Observatory or Lorentz Institute  users plus any packages installed in the virtual environment.
+Note that you do NOT use ''--user'' on the pip command in this case, since that would install in your ''$PYTHONUSERBASE'' directory (see above) instead of the venv!!
+In any cases, it is advisable you keep a backup of your virtual environment configuration by creating a list of installed packages
+<code bash>
+pip freeze > packages.dat
+</code>
+This can help collaborators and fellow developers to reproduce your environment with
+<code bash>
+pip install -r packages.dat
+</code>
+When you are done working in a virtual environment deactivate it running
+<code bash>
+deactivate
+</code>
+At any time, any virtual environment can be destroyed by removing the corresponding folder from the file system so do not panic if things do not work, just delete your virtual environment and start all over again.
+Note: __System administrators will not be responsible and/or manage users virtual environments__. You are strongly advised you consult the documentation.
+==== METHOD 2: OBSOLETE: virtualenv (python 2.x) ====
 This guide refers to virtualenv version 12.0.7.
@@ Line 273: / Line 317: @@
 After starting ''jupyter notebook'' you can select ''python3'' as kernel.
+If you need to work with several python setups (e.g. the system python3, but also from loaded environment modules), it is easy to assign a name to the generated kernel, e.g:
+    python3 -m ipykernel install --user --name system-python3

Computer Documentation Wiki

User Tools

Site Tools

Differences

Page Tools