OREGON STATE UNIVERSITY
Center for Applied Systems & Software ยป

OSU Open Source Lab

Open-CE

The Open Source Lab (OSUOSL) and Center for Genome Research and Biocomputing (CGRB) partner with IBM and OpenPOWER in order to provide a download resources around Open-CE. Open-CE is a community driven software distribution for machine learning that runs on standard Linux platforms with NVIDIA GPU technologies.

Release date: 01/12/2021

What's new

This is release 1.1 of Open Cognitive Environment (Open-CE), code named Meerkat.

  • Added support for CUDA 11.0, which is currently supported on RHEL8.

  • Added recipes for the following new packages: LightGBM, TensorFlow Model Optimization, TensorFlow Addons, PyTorch Lightning Bolts, Python Flatbuffers.

  • Added the open-ce tool for running build and validate commands. This replaces the previously existing build_env.py and build_feedstock.py entry points to Open-CE.

  • Added the open-ce test commands to test packages that are built by Open-CE.

    open-ce build env will now output conda environment files that can be used to create conda environments containing the packages that were just built.

  • The open-ce build image command has been added to create Docker images from the output of open-ce build env.

  • Open-CE build tools can now accept --cuda_versions as an argument to choose a version of CUDA to build with.

  • open-ce build env will now check for circular dependencies between packages.

  • open-ce build env will verify that all packages that are being built can be installed within the same conda environment before starting a build.

  • Added the --skip_build_packages argument to open-ce build env.

  • Jinja can now be used within any Open-CE configuration file.

  • Improved performance when attempting to build packages that already exist.

  • Added the patches key to the Open-CE environment files to allow for patching feedstocks.

Learn more

Get information about planning, configuring, and managing Open-CE 1.1 Below:

Planning

We recommend users use one of the listed operating systems listed below. This is a standard conda repository and can be added to any conda install. Conda must be configured to give priority to installing packages from this channel.

System setup

Open-CE can be installed and run directly on a bare-metal RHEL and Ubuntu based systems.

Supported hardware:

  • IBM Power System IC922 with NVIDIA Tesla T4 GPUs
  • IBM Power System AC922 with NVIDIA Tesla V100 GPUs
  • IBM Power System S822LC with NVIDIA Tesla P100 GPUs
  • x86_64 systems with NVIDIA Tesla V100 or P100 GPUs
  • Supported operating systems:
    • Red Hat Enterprise Linux for POWER LE 7.8
    • Community Enterprise Operating System (CentOS) 7.8
    • Ubuntu 20.04.1
  • Required 3rd party software:
    • NVIDIA GPU driver 440.33 - 460.32
    • CUDA driver 10.2 or 11.0

Installing the Open-CE Repository and Frameworks

Setting up the software repository

The Open-CE MLDL packages are distributed as conda packages in an online conda repository. Conda must be configured to give priority to installing packages from this channel.

Add the Open-CE channel to the conda configuration by running the following command:

conda config --prepend channels https://ftp.osuosl.org/pub/open-ce/current/

Installing frameworks individually

You can install the MLDL frameworks individually. The framework packages include the following versions.

Table 1. Framework packages

Package Description Version Available on ppc64le Available on x86_64
pytorch PyTorch 1.7.1 X X
tensorflow TensorFlow with GPU support 2.4.0 X X
tensorflow-serving TensorFlow Serving 2.4.0 X X
py-xgboost xgboost with GPU support 1.3.0 X X

With the conda environment activated, run the following command:

conda install <package name>

Uninstalling the Open-CE MLDL frameworks

Find information about uninstalling machine learning and deep learning MLDL frameworks.

The MLDL framework packages can be uninstalled individually, or you can uninstall all of the MLDL packages at the same time.

If the frameworks are installed into a separate conda environment, all of the frameworks can be removed by simply deleting the environment:

conda env remove -n <environment name>

Individual frameworks (and any packages that depend on them) can be removed by removing the individual package:

conda remove <package name>

Important: This command removes the specified packages and any packages that depend on any of the specified packages. If you want to skip this dependency checking and remove just the requested packages, add the --force option. However, this may break your environment, so use this option with caution.

Previous releases

We recommend that you install the most current release of Open-CE, however, if you have an earlier version installed, you can find information below:

Previous releases

Open-CE Release 1.0.0

Release date: 11/10/2020

What's new

Open-CE 1.0 is the current release of Open-CE and includes the following features:

  • conda packages are now available on ppc64le.
  • conda packages are now available on x86.
  • TensorFlow 2.3.1
  • PyTorch 1.6.0
  • Open-CE is distributed as prebuilt containers, or on demand through the Conda provisioning process.
    • All of the Conda packages are available in a Open-CE Conda channel
    • Conda packages are available in the Open-CE 1.0.0 Conda channel
    • There is no install package to download, instead connect to the Conda channel and install your packages from there
    • Package dependencies are automatically resolved
    • Delivery of packages is open and continuous
    • Enable Python versions 3.6, 3.7, 3.8
    • You can run more than one framework at the same time in the same environment. For example, you can run TensorFlow and PyTorch at the same time.