hpc.social - Community Syndicated Blog

New Board Members van der Pas and Stotzer

2026-02-16T18:44:35-07:00

The OpenMP ARB welcomes Ruud van der Pas and Eric Stotzer to its Board of Directors.

The post New Board Members van der Pas and Stotzer appeared first on OpenMP.

OpenMP Example Book Survey

2026-02-15T18:47:18-07:00

If you have used our OpenMP API Examples book, take our survey to tell us how we can improve it.

The post OpenMP Example Book Survey appeared first on OpenMP.

New Python Subcommittee, and Anaconda joins OpenMP ARB

2026-02-04T21:00:28-07:00

Anaconda has joined the OpenMP ARB, and the OpenMP ARB has formed a new Python subcommittee to add Python as the fourth supported language in the OpenMP API.

The post New Python Subcommittee, and Anaconda joins OpenMP ARB appeared first on OpenMP.

IWOMP 2026

2026-01-28T17:11:06-07:00

IWOMP 2026 Join us for IWOMP 2026 at Austrian Scientific Computing (ASC) at TU Wien in Vienna (Austria)

The post IWOMP 2026 appeared first on OpenMP.

OpenMP ARB Releases Technical Report 14

2025-11-13T15:53:14-07:00

The OpenMP® ARB has released Technical Report 14, a preview of version 6.1 of the OpenMP API, which will be released in 2026.

The post OpenMP ARB Releases Technical Report 14 appeared first on OpenMP.

Supercomputing 2025

2025-07-29T15:28:58-06:00

November 2025
Visit us in booth 911, attend our BOF, and attend our two tutorials at SC25 in St. Louis. See videos and more.

The post Supercomputing 2025 appeared first on OpenMP.

IWOMP 2025

2025-07-15T00:48:20-06:00

IWOMP 2025 will be held at the UNC Charlotte campus in North Carolina, Oct 1 – Oct. 3, 2025.

The post IWOMP 2025 appeared first on OpenMP.

NextSilicon joins the OpenMP

2025-06-10T10:00:58-06:00

June 10, 2025 — NextSilicon has joined the OpenMP ARB, a group of leading hardware and software vendors and research organizations creating the standard for the most popular shared-memory parallel programming model in use today.

The post NextSilicon joins the OpenMP appeared first on OpenMP.

ISC 2025

2025-02-28T20:12:17-07:00

ISC 2025 will be held in Hamburg, Germany June 10-13, bringing together over 3,500 international attendees to exchange ideas and knowledge.

The post ISC 2025 appeared first on OpenMP.

Status of OpenMP API 6.0 Implementations

2025-02-25T10:00:35-07:00

First implementations of OpenMP 6.0 features are now available in Intel® and GCC compilers.

The post Status of OpenMP API 6.0 Implementations appeared first on OpenMP.

IWOMP 2025

2025-02-15T00:48:20-07:00

IWOMP 2025 will be held at the UNC Charlotte campus in North Carolina, US.

The post IWOMP 2025 appeared first on OpenMP.

OpenMP® ARB Releases OpenMP 6.0 for Easier Programming

2024-11-14T18:00:41-07:00

Develop Parallel Programs Easily and More Control to Developers SC24, Atlanta, Georgia – November 14, 2024 – The OpenMP Architecture Review Board (ARB) is pleased to announce Version 6.0 of the OpenMP API Specification, a major upgrade of the OpenMP language. This new version opens up parallel programming to new applications, makes it easier to [...]

The post OpenMP® ARB Releases OpenMP 6.0 for Easier Programming appeared first on OpenMP.

Supercomputing 2024

2024-08-27T23:13:58-06:00

November 18, 2024
OpenMP will be in Atlanta for Supercomputing 2024 with two tutorials, a BOF, and much more.

The post Supercomputing 2024 appeared first on OpenMP.

OpenMP ARB Releases Public Comment Draft of OpenMP 6.0

2024-08-01T15:54:15-06:00

The OpenMP® Architecture Review Board (ARB) has released Technical Report 13: the final public comment draft of version 6.0 of the OpenMP API. Version 6.0 of the OpenMP API will be released in November 2024.

The post OpenMP ARB Releases Public Comment Draft of OpenMP 6.0 appeared first on OpenMP.

Asynchronous GPU Programming in OpenMP

2024-06-03T21:14:21-06:00

The Centre of Excellence on Performance Optimisation and Productivity published the recording of a webinar on Asynchronous GPU Programming in OpenMP where Christian Terboven and Michael Klemm discuss the optimization of data transfers and asynchronous offloading, hybrid OpenMP and HIP, and advanced task synchronization. Watch now.

The post Asynchronous GPU Programming in OpenMP appeared first on OpenMP.

The OpenMP ARB welcomes new member CEA

2024-05-13T08:00:03-06:00

May 13, 2024 — CEA has joined the OpenMP® ARB, a group of leading hardware and software vendors and research organizations creating the standard for the most popular shared-memory parallel programming model in use today.

The post The OpenMP ARB welcomes new member CEA appeared first on OpenMP.

IWOMP 2024

2024-03-20T14:45:28-06:00

Sept. 23, 2024 The 20th International Workshop on OpenMP - IWOMP is the premier forum to present and discuss issues, trends, recent research ideas, and results related to parallel programming with OpenMP.

The post IWOMP 2024 appeared first on OpenMP.

Looking forward to 2024!

2023-12-22T06:16:33-07:00

Looking forward to next year!

This is my end-of-the year post as we all make our way into the New Year.

We've done quite a lot of things this year to help make oneAPI easier to use - a lot of the blog posts I've written as been towards an eye to educate.

We started off with some blog posts on how to use modern IDEs on Linux to write SYCL code and run them inside a container that we built, making a turn-key effort to build applications.

There was the introduction to 'awesome oneAPI' which showed a set of curated links to oneAPI projects showcasing all the capabilities. We have been updating it regularly, so check it out! We are definitely looking for more AI related projects - are you thinking of a project for next year? Need help? Let me know!

To complement the awesome oneAPI distribution, I have recently launched the oneAPI Web Showcase where we hope to discover upcoming projects that people are working on and showcasing them. There are also links on the website to help you start a project.

We now have community-focused documentation!!

I'll have another blog post up to show how you can help with the documentation by translating the documentation into different languages so that everybody can follow along in their native language. You can see the documentation at https://oneapi-community.github.io/documentation. We hope to grow it into a true community hub for oneAPI and SYCL. Exciting! All of these are community projects in themselves. Want to help out? Reach out or just submit a PR!

Goals for next year

I want to keep building on the work we've done in 2023. So many opportunities!! 2023 was all about meeting developers where they were. Now, it's also time to meet them where they are AND also communicate with them in their own language!!

The key to adopting open platforms is to:

1) Have great documentation that's accessible in as many languages as possible.
2) Plenty of code samples to look at how to do things.
3) Great developer experience - be able to set up your environment and just go!
4) Amazing community that interacts with each other, is active and works together.

These are all totally possible!! But, oneAPI is relatively new and still under one vendor. With the formation of the UXL Foundation, we now have a neutral place for all vendors to congregate and work together. As a community, we should ask our hardware vendors to support level zero and be able to get all the advantages of hardware with a smooth hardware experience.

So where do we want to go from here - here are my personal goals/wish list for next year!

1) reproducible builds - we should be able to continuously build and test oneAPI software.
2) More community assistance in documentation by helping translate the docs we have - as well as having more docs around tips and tricks.
3) Adding more projects to Awesome oneAPI and having more PRs from the community to add their projects! :)

Have a wonderful holiday season and a Happy New Year!

With that, I wish all of you a wonderful holiday season and looking forward to great things in the oneAPI ecosystem in 2024!!

Photo by Jamie Street on Unsplash

Building oneAPI from Source

2023-12-22T06:08:31-07:00

Build oneAPI completely from git

I'm back!! A few raw posts have been languishing and I decided the end of the year was the perfect time to put them out there. This will be one of three (hopefully).

I'm going to focus on how to build oneAPI from git. This is somewhat of a return to my earlier blog post where I talked about how to build the DPC++ compiler and it included the binary versions of the openCL and level zero run time.

That's all well and good but let's consider how we could build the run time from git completely. The ability to do reproducible builds is going to be important later when we dive into buildimg packaging that is up to date and available on any Linux distro.

The build only supports Intel hardware at this point since level zero doesn't support NVidia or AMD GPUs. If you are looking for such support, you might consider Codeplay's plugins that will allow you to use NVidia and AMD hardware.

These blog pages typically only focus on what we can do from an open source perspective and won't really focus on anything that has binary blobs if we can avoid it.

DISCLAIMER: Please don't use this set up for a production environment. It is not well tested. If you find any problems, please reach out in the comments so that I can help debug and update the blog post appropriately.

Setting up the build environment

I like to use containers which makes it easy to quickly set up and automate using distrobox.

You should be able to use whatever Linux distribution you want as long as you can install distrobox. You can, of course, use a virtual machine to accomplish this. I've used Vagrant successfully.

Assuming that you have distrobox installed - let's get to it.

Decide where you want to have the build for instance: ~/src/oneapi-build.

$ distrobox create --image docker.io/library/ubuntu 20.04 --name "oneAPIBuild"
$ distrobox enter oneAPIBuild

You should now be in a container running Ubuntu 20.04.

Install the appropriate packages

$ sudo apt-get install -y build-essential git libssl-dev flex bison libz-dev python3-mako python3-pip automake autoconf libtool pkg-config ruby

You will need a recent version of cmake for the builds. The one that comes with 20.04 is too old.

$  wget https://github.com/Kitware/CMake/releases/download/v3.28.1/cmake-3.28.1.tar.gz
$ tar xvfpz cmake-3.28.1.tar.gz
$ cd cmake-3.28.1
$ ./boostrap
$ ./configure
$ make 
$ sudo make install

Now you should have everything you need for the build.

Understanding the oneAPI Build

There are a number of prerequisites before you start the build. Here is a graphic of how the oneAPI build is put together.

The order of build is:

1) Intel Graphics Engine and its relevant prerequisites which consist of:

Intel Graphics Compiler (igc)
1. SPIRV Headers
2. SPIRV Tools
3. copy of the llvm project
4. vc-intrinsics
5. intel graphcis compiler 2) ocl-icd 3) GMMLib 4) NEO - Intel Compute Runtime - NEO is the primary GPU graphics driver and uses OpenCL to talk to the GPU.
Has the following pre-requisites:
1. Intel graphics compiler (IGC)
2. GMMLib 6) Level Zero 7) DPC++ SYCL Compiler 8) oneTBB Library

That's the progression to get the full build going.

Intel Graphics Engine

Let's start building the first prerequisites for the NEO which is the Intel Graphics Engine:

$ mkdir igc-workspace && cd igc-workspace
$ git clone https://github.com/KhronosGroup/SPIRV-Headers.git --depth 1

$ git clone https://github.com/KhronosGroup/SPIRV-Tools.git --depth 1

$ git clone -b llvmorg-14.0.5 https://github.com/llvm/llvm-project llvm-project --depth 1

$ git clone -b ocl-open-140 https://github.com/intel/opencl-clang llvm-project/llvm/projects/opencl-clang --depth 1

$ git clone -b llvm_release_140 https://github.com/KhronosGroup/SPIRV-LLVM-Translator llvm-project/llvm/projects/llvm-spirv --depth 1

$ git clone https://github.com/intel/vc-intrinsics --depth 1

$ git clone https://github.com/intel/intel-graphics-compiler igc --depth 1

$ mkdir build && cd build
$ cmake ../igc -DCMAKE_INSTALL_PREFIX="/usr/local"
$ make -j `nproc`
$ sudo make install

It should build cleanly. If it doesn't - please check any errors and make sure you have all the prerequisites.

ocl-icd

ocl-icd is an OpenCL loader - and is used to link opencl software when compiling. Make sure you are back in your usual oneapi-build directory.

$ pwd
~/src/oneapi-build
$ git clone https://github.com/OCL-dev/ocl-icd --depth 1
$ cd ocl-icd
$ ./bootstrap
$ ./configure
$ make
$ sudo make install

Install GMMLib

NEO requires GMMLib as one of its prerequisites so we will build that now.

$ pwd
~/src/oneapi-build
$ git clone https://github.com/intel/gmmlib --depth 1
$ cd gmmlib
$ mkdir build
$ cd build
$ cmake .. -DCMAKE_INSTALL_PREFIX="/usr/local"
$ make
$ sudo make install

Install NEO

NEO is the Intel Compute Runtime and is necessary for the SYCL based applications to talk to the GPU. Go back to your ~/src/oneapi-build directory.

$ pwd 
~/src/oneapi-build # please note this output will be different for you
$ mkdir neo-workspace
$ cd neo-workspace
$ git clone https://github.com/intel/compute-runtime neo –depth 1
$ cd neo
$ mkdir build
$ cd build
$ cmake .. -DCMAKE_INSTALL_PREFIX="/usr/local"
$ make
$ sudo make install

Install level-zero

This is the main part of oneAPI and interfaces with NEO or other run times. Since NEO is the only one at the moment - it will only work with Intel devices.

$ pwd
~/src/oneapi-build
$ git clone https://github.com/oneapi-src/level-zero --depth 1
$ mkdir build
$ cd build
$ cmake .. -DCMAKE_INSTALL_PREFIX="/usr/local"
$ make
$ sudo make install

Install the DPC++ Compiler

Now to build the SYCL Compiler.

$ pwd
~/src/oneapi-build
$ mkdir sycl_workspace && cd sycl_workspace
$ export DPCPP_HOME=`pwd`
$ git clone https://github.com/intel/llvm.git -b sycl --depth 1
$ python3 $DPCPP_HOME/llvm/buildbot/configure.py --cmake-opt CMAKE_BUILD_PREFIX="/usr/local"
$ python3 $DPCPP_HOME/llvm/buildbot/compile.py

Install oneTBB

Finally, we need to install oneTBB

$ pwd
~/src/oneapi-build
$ git clone https://github.com/oneapi-src/oneTBB --depth 1
$ cd oneTBB
$ mkdir build
$ cd build
$ cmake .. -DCMAKE_INSTALL_PREFIX="/usr/local"
$ make
$ make install

Set the LD_LIBRARY_PATH

We need to make sure that the linker can find the proper libraries. The easiest way is to either set the LD_LIBRARY_PATH in your .bashrc or put it in /etc/environment.

$ export LD_LIBRARY_PATH="/usr/local/lib"

Test the environment

$ cd ~/src
$ mkdir simple-oneapi-app
$ cd simple-oneapi-app
$ cat > simple-oneapi-app.cpp
#include <sycl/sycl.hpp>

int main() {
  // Creating buffer of 4 ints to be used inside the kernel code
  sycl::buffer<sycl::cl_int, 1> Buffer(4);

  // Creating SYCL queue
  sycl::queue Queue;

  // Size of index space for kernel
  sycl::range<1> NumOfWorkItems{Buffer.size()};

  // Submitting command group(work) to queue
  Queue.submit([&](sycl::handler &cgh) {
    // Getting write only access to the buffer on a device
    auto Accessor = Buffer.get_access<sycl::access::mode::write>(cgh);
    // Executing kernel
    cgh.parallel_for<class FillBuffer>(
        NumOfWorkItems, [=](sycl::id<1> WIid) {
          // Fill buffer with indexes
          Accessor[WIid] = (sycl::cl_int)WIid.get(0);
        });
  });

  // Getting read only access to the buffer on the host.
  // Implicit barrier waiting for queue to complete the work.
  const auto HostAccessor = Buffer.get_access<sycl::access::mode::read>();

  // Check the results
  bool MismatchFound = false;
  for (size_t I = 0; I < Buffer.size(); ++I) {
    if (HostAccessor[I] != I) {
      std::cout << "The result is incorrect for element: " << I
                << " , expected: " << I << " , got: " << HostAccessor[I]
                << std::endl;
      MismatchFound = true;
    }
  }

  if (!MismatchFound) {
    std::cout << "The results are correct!" << std::endl;
  }

  return MismatchFound;
}

$ clang++ -fsycl simple-oneapi-app.cpp -o simple-oneapi-app

When you run the app you should get "Results are correct!".

$ ./simple-oneapi-app
Results are correct!

Now you've successfully built oneAPI from source!

Let me know if you have any issues with the instructions in the comments.

Photo by Dominik Lückmann on Unsplash

Supercomputing 2023 - HPC Social Summary!

2023-11-17T00:00:00-07:00

What a week! We participated in person and virtually for Supercomputing 2023 in Denver. While we cannot cover all the exciting happenings, we will review a brief set of events of interest here.

This year we launched the first HPC Social Noodles Award, a celebration of our frustrations and comical takes on the events of the year. THe top noodle was, of course, the whole CentOS debacle, followed by a few gripes about software and vendors, and the funny noodles starting at item 7 and on.

Beowulf Bash

HPC Social was present (and giving out stickers) at the bash this year! The branding was… excellent.

I Want to Run my MPI

This was a parody music video made by community leader @vsoch to celebrate a generic HPC technology (MPI) in the high performance community!

She made an effort to engage others to participate, and was only moderately successful to get a few shared pictures. It would be a fun idea if others wanted to participate to a greater extent at some future Supercomputing!

Official Greeting

The “official” greeting for SC23 was tapping someone on the shoulder, as announced by HPC Guru.

Texas Tech Tiny Cluster!

Our very own Alan Sill hosted a booth to show up a tiny cluster! While Raspberry Pi clusters have been around for a long time and useful in hobbyist activities, training, home automation, and training, this was one of the first such small clusters running Fedora 39 as a natively installed OS on the head node and Enterprise Linux (in this case Rocky) on the worker nodes. More to come as other mainline popular cluster tools like Warewulf, Spack and/or EasyBuild, and Slurm and/or Flux schedulars are added.

And finally, we close with a few shots shared in the HPC Social slack! We love our community! ❤️

Felix (finally) got his “I am HPC Guru” pin!

Supercomputing 2023

2023-11-12T18:22:52-07:00

November 12, 2023
OpenMP will be in Denver for Supercomputing 2023 with four tutorials, a BOF, and more.

The post Supercomputing 2023 appeared first on OpenMP.

OpenMP ARB Releases Technical Report 12

2023-11-09T07:30:51-07:00

The OpenMP® Architecture Review Board (ARB) has released Technical Report 12, the second preview of version 6.0 of the OpenMP API, which will be released in 2024.

The post OpenMP ARB Releases Technical Report 12 appeared first on OpenMP.

Fortran Package Manager and OpenMP

2023-10-11T17:35:25-06:00

The Fortran Package Manager, or fpm, is a community-driven, open-source build tool and package manager for the Fortran language. fpm makes it easy for beginners to develop applications. It streamlines project setup by quickly and easily generating Fortran project templates, facilitating rapid prototyping.

The post Fortran Package Manager and OpenMP appeared first on OpenMP.

oneAPI moves to Unified Acceleration Foundation (UXL)

2023-09-30T03:36:38-06:00

As accelerators have become more prevalent in the industry there has been a bifurcation in the industry which typically resolves itself. Initially there has been many efforts by a myriad of vendors in the GPU accelerator space with various degrees of openness.

For a long period of time, incubation of the oneAPI ecosystem started at Intel through both DPC++ SYCL compiler, and the publishing of open specifications of oneAPI and their open source implementations. While the messaging around the specs were that they were open to all contributors – it could be hard to feel comfortable especially for competitors to enter spaces where the perception is that it isn’t a true neutral space.

With oneAPI spec’s now under the aegis of the Linux Foundation the spec and the open source implementations will be well managed under established norms. There is now a true center of gravity to work together as equal partners on the oneAPI spec and their open source implementations.

We can now focus on driving a true industry driven standard on heterogeneous computing under the UXL Foundation and have some serious collaboration to finally use all your hardware. This will herald a sustainable ecosystem that we can all be proud of.

There are still challenges going forward. How our toolchains work together under UXL Foundation is going to be important going forward. We can address these concerns by vigorously participating in the UXL Foundation. Creating an open ecosystem is always challenging as many many partners need to agree and align on goals and processes. Listening to each other and the community is going to be key going forward.

With all that being said - I hope that you will take the time to look at what has been established so far. We have a humble beginning but with your help and participation we can take it to the next level. For further reading, please see https://uxlfoundation.org/. Looking forward to seeing you all there.

Cover image: Photo by Scott Blake on Unsplash

IWOMP 2023

2023-09-12T21:35:36-06:00

Sept. 12, 2023 The 19th International Workshop on OpenMP - IWOMP is the premier forum to present and discuss issues, trends, recent research ideas, and results related to parallel programming with OpenMP.

The post IWOMP 2023 appeared first on OpenMP.

Migrating reduction operations to SYCL in a molecular docking application

2023-08-10T22:30:31-06:00

I completed porting of a molecular docking application from CUDA to SYCL using the Intel® DPC++ Compatibility Tool (Compatibility Tool) in June 2021. Let me share selected techniques that I used without delving into the details of the docking application. If you want to learn how to use this tool to migrate CUDA applications to SYCL, please refer to [1].

The Compatibility Tool adds comments in the code where manual migration may be required. Typically, the manual changes required fall into two categories. First, changes are required for the code to compile and make the code functionally correct. Other changes are necessary to get better performance. Here, I will cover code that uses the operation of 'reduction'. Reductions are frequently used in High Performance Computing and scientific applications and can be performance hotspots. The first example finds the sum of integers and the second finds the minimum of floats and the identifier of the run that corresponds to the minimum.

Integer Reductions to find the number of evaluations

The docking application performs integer reductions to keep a running count of the number of score evaluations. This reduction is implemented as a multi-line macro in CUDA as shown below.

#define REDUCEINTEGERSUM(value, pAccumulator)     
    if (threadIdx.x == 0)     
    {     
        *pAccumulator = 0;     
    }     
    __threadfence();     
    __syncthreads();     
    if (__any_sync(0xffffffff, value != 0))     
    {     
        uint32_t tgx            = threadIdx.x & cData.warpmask;     
        value                  += __shfl_sync(0xffffffff, value, tgx ^ 1);     
        value                  += __shfl_sync(0xffffffff, value, tgx ^ 2);     
        value                  += __shfl_sync(0xffffffff, value, tgx ^ 4);     
        value                  += __shfl_sync(0xffffffff, value, tgx ^ 8);     
        value                  += __shfl_sync(0xffffffff, value, tgx ^ 16);     
        if (tgx == 0)     
        {     
            atomicAdd(pAccumulator, value);     
        }     
    }     
    __threadfence();     
    __syncthreads();     
    value = *pAccumulator;     
    __syncthreads();

Let us review what this code is doing:

The code is called for each work item (thread) in a work group (warp)
*pAccumulator is where the final sum is stored summing across all work items
The combination of __threadfence() and __syncthreads() guarantees memory consistency and synchronizes threads in the warp at the point of the call.
The __any_sync() call executes the block for those non-exited threads for which 'value != 0'
The following __shfl_sync calls do a tree-wise summing with the final sum available in the first thread in the warp in variable value
The value is then added to the Accumulator atomically with atomicAdd and finally all threads assign the sum to the value variable.

For more details about these CUDA calls please refer to [2].

The Compatibility tool was not able to automatically migrate this code with the following comments.

/*
DPCT1023:40: The DPC++ sub-group does not support mask options for sycl::ext::oneapi::any_of.

DPCT1023:41: The DPC++ sub-group does not support mask options for shuffle.

DPCT1007:39: Migration of this CUDA API is not supported by the Intel(R) DPC++ Compatibility Tool.
*/

However, SYCL supports a rich set of functions for performing reductions. In this case, the reduce_over_group() function in SYCL can be used to create the same functionality as the above code as follows.


#define REDUCEINTEGERSUM(value, pAccumulator)     
        int val = sycl::reduce_over_group(item_ct1.get_group(), value, std::plus<>());      
        *pAccumulator = val;     
        item_ct1.barrier(sycl::access::fence_space::local_space);

The sycl::reduce_over_group is a collective function. The usage of this function simplifies the macro. The function takes the group, the value to be reduced, and the reduction operation which in this case is plus or summation. The function can adapt to varied sizes of work groups in SYCL and will use the best available optimizations available per the compiler and run-time.

Finding the minimum energy

In another part of the application, a block of CUDA threads perform shuffles to find the minimum of scores v0 and the corresponding identifier k0 of the run in the simulation that is the minimum score. The CUDA code calls a macro WARPMINIMUM2 (not shown) which in turn calls another macro WARPMINIMUMEXCHANGE (shown) with mask set to 1, 2, 4, 8, and 16.

#define WARPMINIMUMEXCHANGE(tgx, v0, k0, mask)     
    {     
        float v1    = v0;     
        int k1      = k0;     
        int otgx    = tgx ^ mask;     
        float v2    = __shfl_sync(0xffffffff, v0, otgx);     
        int k2      = __shfl_sync(0xffffffff, k0, otgx);     
        int flag    = ((v1 < v2) ^ (tgx > otgx)) && (v1 != v2);     
        k0          = flag ? k1 : k2;     
        v0          = flag ? v1 : v2;     
    }

The __shfl_sync provides a way of moving a value from one thread to other threads in the warp in one instruction. In this code snippet __shfl_sync gets the v0 or k0 value from the thread identified by the otgx mask and saves it in v2, k2 variables. We then compare v1 with v2 to set flag and eventually store the minimum in v0 and the run identifier for this minimum in k0.

Compatibility Tool could not completely migrate this code and included this comment as the reason it could not. However, Compatibility Tool correctly replaced the __shfl_sync call with SYCL shuffle call as shown in the below diff which shows the manual change.

/*
DPCT1023:57: The DPC++ sub-group does not support mask options for shuffle.
*/

This comment indicates that the shuffle call in SYCL does not use a mask as shown below.

#define WARPMINIMUMEXCHANGE(tgx, v0, k0, mask)     
        {     
                float v1 = v0;     
                int k1 = k0;     
                int otgx = tgx ^ mask;     
-               float v2 = item_ct1.get_sub_group().shuffle(energy, otgx);     
+               float v2 = item_ct1.get_sub_group().shuffle(v0, otgx);  
-               int k2 = item_ct1.get_sub_group().shuffle(bestID, otgx);     
+               int k2 = item_ct1.get_sub_group().shuffle(k0, otgx);     
                int flag = ((v1 < v2) ^ (tgx > otgx)) && (v1 != v2);     
                k0 = flag ? k1 : k2;     
                v0 = flag ? v1 : v2;     
        }

In this case, Compatibility Tool performed incorrect variable substitution for v0 and k0 in the shuffle calls using energy and bestID variables from the caller function. We manually fixed this by replacing energy with v0 and bestID with k0. This bug has been fixed in recent versions of the Compatibility Tool.

Summary

In summary, reduction operations in CUDA applications may not be migrated correctly by the Compatibility Tool. Review the comments provided by the tool to understand if manual migration is necessary and what change might be required. A good understanding of the original CUDA code will then help to make manual changes to develop functionally correct code in SYCL.

[1] https://www.intel.com/content/www/us/en/docs/dpcpp-compatibility-tool/get-started-guide/2023-1/overview.html

[2] https://developer.nvidia.com/blog/using-cuda-warp-level-primitives/

IWOMP 2023

2023-07-17T21:35:36-06:00

Sept. 12-15, 2023 The 19th International Workshop on OpenMP - IWOMP is the premier forum to present and discuss issues, trends, recent research ideas, and results related to parallel programming with OpenMP.

The post IWOMP 2023 appeared first on OpenMP.

Supercomputing 2023

2023-07-17T18:22:52-06:00

November 12-16, 2023
OpenMP will be in Denver for Supercomputing 2023 with four tutorials, a BOF, and more.

The post Supercomputing 2023 appeared first on OpenMP.

Awesome oneAPI announced!

2023-06-22T20:37:53-06:00

One of the ways that I feel we should engage when building community for
a relatively new ecosystem like oneAPI is the ability to showcase it's
capabilities. We know that AI especially generative AI have captured
the hearts and minds of many especially being able to build interesting
visuals using prompts. As well, GPU offloading, and learning algorithms
also have gained traction in this space as well.

We talk about the democratization of
AI
and its important in the emerging new chapter of AI. There is no better
time and greater need to build things on an open spec'd platform where
it's clear what is going and out and that the conversations are public,
the decisions are public.

An open spec'd platform is no good if it doesn't show
utility. Developers aren't going to consume your platform if you
cannot show viability, ease of use, and performance. So to show utility,
we worked on building a curated list of projects that show examples of
the utility of oneAPI, but also provide projects that are worth spending
time being part of. Some of these projects might surprise you in that
they use oneAPI like Blender.

Some of you might have heard of the 'awesome' lists
concept which are lists of github repos that are great
examples of using that subject matter. For instance, Awesome
Pytorch is a good
example of a list that shows cool projects that use pytorch. The list
is meant to be simple, developer friendly, and easy to navigate.

We were inspired by these lists as they seem like a great way to find your
way to good projects. So we created our own 'awesome' list of projects
for SYCL and oneAPI. So without further ado, feel free to check out
awesome oneAPI. We
would of course love feedback and if you have projects that might fit
this list, please fork and submit a PR! Questions are welcome, you can
reach me on mastodon at @sri@mast.hpc.social or if you're on dev.to -
just hit the comments!

The hpc.social project - talk at EasyBuild Users Meeting

2023-05-01T00:00:00-06:00

In case you missed it, Alan and vsoch presented the hpc.social at the annual EasyBuild User’s meeting:

In this talk they share the origins of the project, the current project, and prompt for questions or ideas about desire for the future. As a result of the talk, the hpc.social events page has already been refactored to properly show the HPC Huddle community feed, and other communities that have ical feeds are welcome to contribute their event feeds there.

OpenMP® ARB adds new member Samsung

2023-04-13T12:00:36-06:00

The OpenMP Architecture Review Board (ARB) today announced that Samsung has joined the board.

The post OpenMP® ARB adds new member Samsung appeared first on OpenMP.

Using oneAPI AI Toolkits from Intel and Accenture Part 2

2023-03-31T23:21:33-06:00

Introduction

This is part 2 of our blog series. These posts are really about inspiring others to think of cool projects to do with oneAPI. In the last blog post, we discussed several toolkits that I thought were interesting. If someone produced an idea that uses those toolkits – I would love to know!!
In this blog post, I want to focus on two other AI toolkits and how to use them.
There is one other aspect that we should consider when looking at these toolkits: what do you believe might be unintended consequences?? As an exercise, I am going to go over these toolkits, but I would love to hear what you might believe are unintended consequences that have the potential to be overlooked.

Personalized Retail Experiences with Enhanced Customer Segmentation

Accenture has over 30 toolkits to showcase oneAPI, with more to come. In this post, I am going to look at this idea of using AI to personalize your shopping experience. I think all of us who do any kind of online shopping know the importance of providing a personalized shopping experience. First, we must understand how one might implement a personal shopping experience.
Today, retailers have an incredible amount of data at their disposal. The analytics market is worth up to $20 billion around the world and is growing at a 19.3 percent Compound Annual Growth Rate (CAGR). Retailers are eager to understand customer behavior so that they can provide a better shopping experience and thus drive brand loyalty.

To access the AI toolkit clone the github repository:

$ git clone https://github.com/oneapi-src/customer-segmentation

The reference kit will show how to analyze customer purchasing data and segment it into clusters based on customer behavior. It will also show you how to optimize the reference solution using the Intel Scikit-Learn extension.

The reference example uses an experimental dataset. The dataset is a set of 500k transactions covering 4000 customers from a UK multinational online retailer over a period of a year. The dataset is fed into KMeans and DBSCAN algorithms to label cluster based on different customer behaviors.

Try it out and send me some feedback. Also, keep in mind the challenge of what could go wrong. (disclaimer: I don’t know myself - I’m curious to hear theories)

Faster session Notes with Speech-to-Text AI for Healthcare Providers

My second example is going back to healthcare. Always a fun one. The same challenge as in the previous one.

The premise for this is that mental health providers are required to document their sessions using progress notes. These recorded sessions then need to be transcribed into written notes, and be stored for later reference.

Managing these notes can take quite a bit of time. The idea, then, is to take these recorded notes and feed them to a speech-to-text AI algorithm and provide a summary. This summary can then be used to coordinate care, creating a paper trail, compliance, and keeping track of the state of the client.

By reducing the book keeping, a therapist would have more time for their patients or the capacity to see more patients. Given the shortage of mental health professionals, being able to be more efficient and allowing more “human”contact time will help mental health professionals provide their clients with better care.

You can find the code for this implementation at:

$ git clone https://github.com/oneapi-src/ai-transcribe

The high level overview of this implementation is something like this. The conversion from speech to text is achieved by using a sequence-to-sequence framework called Fairseq. Sequence-to-sequence modeling is a type of machine learning that is commonly built to create summaries, text translations and so on. It was initially conceived by Google. Fairseq is an open source sequence-to-sequence framework from Facebook.

The process is described like this:

Take your dataset of unstructured audio samples
Run it through a data preprocessing using Fairseq modeling
Using GAN you create a trained model using both the training data and the pre-processing data.
Apply inference to generate the text.

I think one of the more interesting parts of this pipeline is the GAN - which is described as two algorithms: one as a generator and the other as a test. The two work against each other until they both end up with the same dataset that, ostensibly, is accurate.

One other piece that is missing as part of training the algorithm is a database of English text corpus data. This database contains speech audio files and text transcription. It is used to create a relationship between an audio signal and phonemes as part of speech recognition.

Where GAN comes in is that a neural network is trained to generate what it thinks are the representations of the phonemes as opposed to the real world data obtained from the corpus data. The other neural network is trained on the corpus data and acts as the validator - as the two neural networks work with each other - the generator portion of the neural network will finally produce the results as expected by the other neural network.

It is through this that we can validate that the output is correct.

The entire software to do this is all open source - I would love to hear from people who have tried it and share what their results were!

I think it would be interesting to train this with humans to determine how accurate it is so you can fully train the corpus and generator algorithm for better results.

Summary

I’ve reviewed two Accenture toolkits that demonstrate how AI can be used practically with real examples. Being a newcomer in this area, there is so much that I don’t know. Ironically, I use chatGPT to help explain some of the salient bits about how GAN works vis-a-vis audio data to really understand what was happening, especially in regards to mapping with words and phonemes.

Looking forward to people’s responses to this post and enjoying a great conversation about AI, its potential uses and applications by using real world examples.

Call to Action

Has this blog post inspired you to write something based on the oneAPI AI toolkits? Let me know - I would love to know how it works out for you!

Training AI with oneAPI part 1

2023-03-30T05:54:13-06:00

oneAPI and AI

My last few blog posts were pretty fun to write. I'm going to change subjects today and talk about AI. We'll learn about what resources are around for those of you who know the particulars about AI and want to start a project, but are interested in delving deeper into the capabilities.

Accenture and Intel

It should be no surprise that Intel is the largest stakeholder in oneAPI, if you've done any kind of research on oneAPI at all. Intel's interest is creating a performative software stack that runs well, not just on Intel platforms, but on any platform. oneAPI's purpose is to be able to take advantage of all the hardware you have on your system and not just the GPU or CPU.

Let's talk about oneAPI and the AI toolkits that have been released over the past 8 months or so. Intel worked with Accenture to release some open source "recipes" for AI. If you've always wanted to delve into AI but have problems on starting a project - this is a great place to ponder what kind of a project you'd want to write based on what problems it solves. These AI kits are a one-stop shop of everything you'd need to get started, including the training data!

oneAPI AI Toolkits

The Intel and Accenture tool kits are all released as open source and can be found on github. The tool kits span a number of industries from telecommunications to health & life sciences to retail and more. They provide a great showcase on how AI is being used today in different industries and what problems they are solving.

The best part of these tool kits is how comprehensive they are. You'll have everything you need to get the toolkit working smoothly, including the source code and training data.

In this blog post, I'll focus on two reference kits and explain how they work. The github page for them is fairly self-explanatory but it's still worth going over them.

Disease Prediction

The first reference kit is disease prediction. This toolkit will look through patient records and look for a possible disease indication. The interesting part of this is the use of NLP (Natural Language Processing) to sort through unstructured data located in patient records.

NLP has been used by healthcare for quite some time - but only now has there been significant investment by healthcare payers. NLP can be used in other areas as well. For instance, understanding what kind of dosage of medication that a patient should take based on their particular genetics! By training on the data of millions of patients, one could really understand the unique properties of the health of individuals and act accordingly.

There is also an interesting social change - the ability to objectively look at patients and their needs, especially women's health needs, means that we can create prediction models based on symptoms that can be followed up on. It's possible to reduce bias in care through such a system - as long as the AI employed is not biased.

To actually pore through patient records, you'd need an NLP that is already pre-trained on reading words. Normally, you'd have to have an algorithm and then use a large number of data sets to get to a point where you'd be able to read natural language. This is why they use the BERT language model. More accurately, a specialized form of BERT called clinicalBERT which includes clinical jargon and medical references.

If you've followed the text of the github repo - the process is pretty simple.

You have the clinicalBERT language model which you would use to train your AI using the data from clinical records.
The language model then creates an association with symptoms to predicted disease probabilities.
Now you have a model that you can apply to any set of symptoms with an output of predicted probabilities.
The process of applying data to a model is called model inference

We won't go through the process of building the software and training it here. The repo has some clear steps in how to train your model.

The prerequisites require Python and PyTorch v1.11. The repo also goes on to describe how to optionally use the Intel extensions for Python for better performance. I'd like to also add that these extensions exist temporarily while the process of upstreaming to main line python continues. Rather than wait, the community can enjoy the optimizations now, rather than at some future date.

There are some instructions to do some bench marking - if you're interested in seeing how well it works on various other platforms.

Increase Mortgage Loan Default Risk Prediction Speed

Next, I'll focus on banking and loans. Banks use AI prediction models to determine risk. This is an interesting case study and I'd like to see some comments about this particular scenario because I expect that some of you will have opinions!

The problem statement here is that in Q4 2021, mortgage delinquencies were 4.65% and outstanding balances of unpaid principals was approximately $2.6 trillion dollars. The average time to complete a foreclosure process was 941 days, leading to a result of approximately $7.6 billion in foreclosure costs alone.

What this kit offers is the ability to manage default risk, handle larger data sets, and reduce the underwriting wait time. The kit will improve customer service quality and speed up loan processing.

So, let's take a look at the github repo for Loan Default Risk Prediction using XGBoost.

This reference solution follows a similar idea. XGBoost XBoost is part of a family of machine learning algorithms that uses decision trees. Specifically, XGBoost uses gradient boosting which instead of using one decision tree, it uses an ensemble of decision trees. A decision tree is nothing more than a model that tries to make a prediction basted on a selection of data.

With that background in mind, you can look at the intended data set that is being used with the kind of parameters.

To make it even more interesting, a modification was made to the data set by adding synthetic bias_variable. The idea is to add bias value for each loan - the value is generated randomly. The reason is to demonstrate bias between a protected class and a privileged class. It isn't used as part of training the model.

Another thing that's wonderful about this AI toolkit is how it demonstrates bias model. The section about "Fairness Evaluation" goes into some length about whether the algorithm is fair. This is an important consideration when training AI models and is an area of active research. While AI can be a powerful tool, it can also be a tool that can augment inequalities and inequities and, when used in decision making, can exacerbate and preserve these existing inequalities.

To use the toolkit, you'll need Python v3.9 and XGBoost v0.81 and clone the repo:

git clone https://www.github.com/oneapi-src/loan-default-risk-prediction

use the bash script to set up the environment.

Follow the instructions in the repo on how to get the model running.

I will leave it up to the reader to run this model. Specifically run it many times and observe the fairness metric and see how that changes.

Call to Action

What do you think about the ease of using these tool kits so far? I would love to hear your thoughts and see if the results were intriguing to you. The bias factor in the last AI toolkit is something that intrigues me - is the model of fairness really fair? How would you change any of these models?

Hit me up on the comments, let's have a conversation!

SiFive joins the OpenMP® effort

2023-03-14T09:00:24-06:00

SiFive joins the OpenMP Architecture Review Board (ARB), a group of leading hardware vendors, software vendors and research organizations, in creating the standard for the most popular shared-memory parallel programming model in use today.

The post SiFive joins the OpenMP® effort appeared first on OpenMP.

Embedded Systems and the OpenMP® API

2023-03-14T09:00:22-06:00

Embedded systems are used in fields as diverse as telecommunication systems, robotics, automotive, and medical applications. They are very heterogeneous and consist of multicore systems and accelerators.

The post Embedded Systems and the OpenMP® API appeared first on OpenMP.

Modern Software Development Tools and oneAPI Part 3

2023-02-28T02:19:25-07:00

This is the third part in the series. Part 1 is here and Part 2 is here.

Welcome to the third, and likely the final, post in this blog series! To recap, in the last blog post we talked about build systems particularly meson Before that, we talked about building a container that contains the pure open source elements of the oneAPI developer environment and use it to build a simple oneAPI SYCL program.

In this post, we're going to take our key learnings from the last two blog posts and use them to build a true user-friendly experience where you can write code using a modern IDE and compile and run them inside a container like you might be used to on other platforms like Windows and MacOS.

First, allow us to introduce you to this modern IDE - 'GNOME Builder'. GNOME Builder is an IDE developed for GNOME desktop. It is integrated to be able to write GNOME and GTK applications easily with all the modern features one would expect from a IDE, and then some.

It has an impressive set of features - the author, Christian Hergert, wrote it because he was [frustrated (https://foundation.gnome.org/2015/01/09/interview-with-christian-hergert-about-builder-an-ide-for-gnome-2/) with the state of IDEs on the Linux platform. GNOME Builder is not just an IDE, but a complete showcase of what a non-trivial application written in GNOME can do.

This blog post is about oneAPI - why use an IDE that is optimized for using GNOME to build applications?

Great question. The desktop ecosystem (GNOME and KDE has been focused on distribution of apps through a container technology called flatpak. Flatpak allows you to have an runtime that contains everything to run a GNOME (or KDE) application. There is an associated SDK that contains all the tools needed to build the application. GNOME Builder is the first IDE that integrates this idea of containerized applications into the user experience. With Builder, you only need the application - you don't need a compiler, profiler, or development libraries - it integrates all that inside a container. This means that you don't need to think about how to setup a developer environment for any GNOME application.

The containers in the past have been flatpak based containers. But it turns out that you can leverage GNOME Builder to use any container created by podman, toolbox, or distrobox.

In essence, the first blog post in this series mimicked what flatpak already does: which is a container that contains everything you need to build an oneAPI application/program instead of a GNOME one.

In a bit of circularity that you might find amusing - we will use flatpak to get the application and then use another comtainer to build our sample application that uses Meson.

If you have not read the first two blog posts, this might be a good time to stop and read those first because we'll be using the container we created in the first blog post and the build system we used in the second blog post. It's also important you use a distro like Fedora or openSUSE that supports flatpak out of the box.

With the pre-requisites out of the way, let's first start by installing GNOME Builder. You can use any desktop you want, but I will be using GNOME here as it is what I usually run, please translate accordingly.

Here are the steps:

First make sure you add the flathub flatpak respository:

$ flatpak remote-add --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo

Install GNOME Builder:

$ flatpak install flathub org.gnome.Builder

Run GNOME builder either through your desktop launch options. For GNOME, hit the meta key (usually Windows key) and then type in "Builder" - GNOME Builder should be your first, and likely only, option. You can also run it from the command line:,

$ flatpak run org.gnome.Builder

You should now have GNOME Builder running on your machine!

Creating a Project

The first step is to create a project.

Select "Create New Project..."

You will be presented with a new screen where you put in the details for the project. Let's call our project "oneapi-simple".

Next we need to select the application-id. Application-ids are generally a reverse DNS type of string usually based on a hostname. I have my own domain, so I usually use that. But you can use whateer you like. In this case, I am going to use me.ramkrishna.oneapisimple.

We want to use C++, so under Language change it to C++. Note that the Template section has now changed to 'Command Line Tool' Which is exactly what we want.

Here is a filled-out screenshot of the window from Builder:

We now create the project! Selected the "Create Project" and we are now ready to continue.

GNOME Builder has two sections - the sidebar and the main editor window. The side bar will have our files and so click on "src" and you should see two files - main.cpp and meson.buid.

Setup the build system

You will notice that the project is already set up to use meson by default. Meson is the preferred build system for GNOME. Meson was created by someone from the GNOME community and thus is already well trusted.
In the application space, meson has proven to be quite popular replacement for autotools.

Let's leave main.cpp alone for now, and focus on meson.build. If you click on meson.build, you'll see that it looks like this:

oneapi_simple_sources = [
  'main.cpp',
]

oneapi_simple_deps = [
]

executable('oneapi-simple', oneapi_simple_sources,
  dependencies: oneapi_simple_deps,
  install: true,
)

This meson.build is set up to compile a generic project with the g++ compiler. So that's not going to work. If you read the previous blog post, we went through what we would need to make it work with the SYCL compiler.

Replace the contents of meson.build with this:

simple_oneapi_sources = files('main.cpp')

simple_oneapi_deps = [
]

executable('simple-oneapi', simple_oneapi_sources,
  link_args:'-fsycl',
  cpp_args:'-fsycl',
  dependencies: simple_oneapi_deps,
  install: true, install_dir: '/var/home/sri/Projects/oneapi-simple/bin'
)

For the SYCL compiler, we need some extra linker flags. We're actually missing something even more important and that's the setup for the compiler itself!

Click on the 'meson.build' file in the top level - which should be right next to the 'COPYING' file. You'll notice that every time you open a new file, it creates a new tab in the editor view. You can easily switch to each file by clicking on the tab.

Let's take a look at it. It should look like this.

project('oneapi-simple', ['cpp', 'c'],
          version: '0.1.0',
    meson_version: '>= 0.59.0',
  default_options: [ 'warning_level=2', 'werror=false', 'cpp_std=gnu++2a', ],
)

subdir('src')

The important part here is that we are identifying that this project is C++. All of this is correct and there is nothing more to be done.

Set up our source

Now, that we have the build set up. It's time to replace the code in main.cpp. Currently, the code looks like:

#include <iostream>

int main() {
    std::cout << "Hello World\n";

    return 0;
}

We are going to replace it with:

#include <sycl/sycl.hpp>

int main() {
  // Creating buffer of 4 ints to be used inside the kernel code
  sycl::buffer<sycl::cl_int, 1> Buffer(4);

  // Creating SYCL queue
  sycl::queue Queue;

  // Size of index space for kernel
  sycl::range<1> NumOfWorkItems{Buffer.size()};

  // Submitting command group(work) to queue
  Queue.submit([&](sycl::handler &cgh) {
    // Getting write only access to the buffer on a device
    auto Accessor = Buffer.get_access<sycl::access::mode::write>(cgh);
    // Executing kernel
    cgh.parallel_for<class FillBuffer>(
        NumOfWorkItems, [=](sycl::id<1> WIid) {
          // Fill buffer with indexes
          Accessor[WIid] = (sycl::cl_int)WIid.get(0);
        });
  });

  // Getting read only access to the buffer on the host.
  // Implicit barrier waiting for queue to complete the work.
  const auto HostAccessor = Buffer.get_access<sycl::access::mode::read>();

  // Check the results
  bool MismatchFound = false;
  for (size_t I = 0; I < Buffer.size(); ++I) {
    if (HostAccessor[I] != I) {
      std::cout << "The result is incorrect for element: " << I
                << " , expected: " << I << " , got: " << HostAccessor[I]
                << std::endl;
      MismatchFound = true;
    }
  }

  if (!MismatchFound) {
    std::cout << "The results are correct!" << std::endl;
  }

  return MismatchFound;
}

OK - now we have everything. But we can't quite compile yet. Right now, if you tried to compile this - it won't work. The reason is, the build is currently set up for native build'. Which means it will try to use the toolchain on the host system. On the host system, we don't have any of the oneAPI libraries or the SYCL compiler. So it won't find anything. Everything we wanted is encapsulated in a container.

This is why GNOME Builder is especially suited to do this exercise on Linux because you set the run and build environment to any podman (or docker) container.

Set the build and run environment to our SYCL container.

Refer to the
first
blog post on how to setup the build and run container.

In that blog post, we named our container - 'oneapi'. It should container the SYCL compiler that we
built and all the accompanying libraries to build our simple SYCL program.

To set the build type - we need to move our cursor to the widget at the top in the center next to the
hammer icon. Click on the down arrow, and then select 'Configure Project', there is a keyboard shortcut
"alt+," (hold alt and then comma) and the window should pop up.

Select "Default" at the bottom of the dialog box.

Under Build Environment, you want to change that from 'Host Operating System' to 'oneapi'. If 'oneapi',
does not appear on your list of choices then you have not created the container using distrobox. You
should refer to the first blog post in the series for testing.

At this point, we have our build system using our container - but we aren't done yet. The problem now is
that the build system will explicitly use c++ instead of the SYCL compiler. To override using the native toolchain, we generally use an environmental variable. This is generally not recommended but for sake of simplicity, we will use it for now. In another blog post, we can revisit the issue. For the impatient, it requires that you use the native file feature of meson - see https://mesonbuild.com/Machine-files.html.

For now, we will use Builder's ability to set shell environment variables to set the CXX and other
critical environment variables.

Click on 'Add Variables' and set the following key value pairs (be sure to replace the paths to the
correct paths):

DPCPP_HOME=/var/home/your_login/src/dpcplusplus
PATH=/usr/bin:/usr/sbin:/usr/local/bin:/home/yourlogin/bin:/home/yourlogin/.local/bin:/var/home/yourlogin/src/dpcplusplus/llvm/build/bin
LD_LIBRARY_PATH=/var/home/yourlogin/src/dpcplusplus/llvm/build/lib
CC=clang
CXX=clang++

Your config should look like this:

The environment variables that are set are mirrored from the environment variables we had to set when we set up a simple oneapi codebase inside the container in the first blog post. We are merely recreating it.

At this point, you can click on the "hammer" icon and GNOME Builder should proceed to properly build the source code. It will give two warnings that you can safely ignore at this point.

To execute the program, you need to hit the right pointing triangle(it looks like a "play" button) and it will try to execute it.

You'll note that it was not able to execute.

That's becasue when it is running it doesn't set the LD_LIBRARY_PATH
inside the container. Since build environment is using non-standard paths we have to do a trick to set everything up so that it can find the libraries it needs.

So, to mitigate that we need to create a wrapper script that will set the LD_LIBRARY_PATH before executing. In another blog post, we will work on something a litte more clever. This will do for now.

Let's call the script 'run-oneapi.sh'. Here is the very simple code for it:

#!/bin/sh
export LD_LIBRARY_PATH="/var/home/sri/src/dpcplusplus/llvm/build/lib"
exec /var/home/sri/Projects/oneapi-simple/bin/oneapi-simple

Install it somewhere within your PATH environment. I have mine in ~/Projects/oneapi-simple/bin where the
run time binary gets built and installed.

Once you have that, you need to let builder know how to run it.

The first step is to go back to the build configuration menu, use the keyboard shortcut ALT-, and then select "Command" on the far left column.

Select "Create Command" and then fill in the dialog box like this:

Once you've added that, you are ready to configure the run command to use this script.

Click on 'Applications'

On the first line you'll see "Run Command" which will be set to "Automatically Discover". Use the drop down list to select "Run-oneapi".

Close the dialog box, and you will now have setup Builder to build and run. Since, everything is already cached. You will need to re-run the build.

Select the drop down list next to the hammer icon and select "Rebuild". This will rebuild the source from scratch and clear out all the cache.

You will now be setup to run.

Click on the play icon next to the hammer icon and it should now properly build and run.

Congratulations - you have now succesfully set up building an oneAPI build on GNOME Builder.

There are a lot of ways to go from here. I would love to hear if anybody actually set this up and give some feedback on whether you were able to make this work and what further plans you have.

There are definitely some improvements that need to be done. Since this set up doesn't actually work to ship an application.

This ends third in the series. I might revisit. I would love to get feedback, improvements and whether you all are hacking code using GNOME Builder!

2023 ECP COMMUNITY BOF DAYS

2023-02-14T14:19:57-07:00

The Exascale Computing Project (ECP) 2023 Community Birds-of-a-Feather (BOF) Days will take place February 14–16, with multiple sessions each day.

The post 2023 ECP COMMUNITY BOF DAYS appeared first on OpenMP.

Modern Software Development Tools and oneAPI Part 2

2023-01-10T23:08:36-07:00

Modern Software Development Tools and oneAPI Part 2

This is part 2 of using a modern open source toolchain to build oneAPI based applications. If this is the first time you're seeing this post, you can read part one here

In part one, we talked about how easy it is to put a container together that will allow you to have everything you need to compile and build an oneAPI application. However, the sample code was compiled simply with:

clang++ -fsycl simple-oneapi.cpp -o simple-sycl

Not particularly interesting, is it?

This post will focus on building this same code using Meson. Meson focuses on simplicity, is a build system generator, but has a concept of back-ends where it can generate whatever the back-end defines. Together with the backend creates a featureful build system. Currently, the default back-end is the fabulous ninja on the Linux platform. Ninja is a command runner that is extremely fast compared to something like Make. Meson also supports xcode and vscode as back-ends allowing to easily use Meson on MacOS and Windows.

Meson just recently hit 1.0 after ten years of development. A wonderful milestone. You can read about it in this blog post.

Installing Meson and Ninja

To get Meson working, we first need to install its prerequisites. This is fairly easy to do, assuming that you are not in the oneAPI container.

$ distrobox enter oneapi
$ pip3 install --user meson
$ pip3 install --user ninja

You are, of course, welcome to install them site-wide so you don't clutter up your home directory and also, have a clear delineation between toolchain in the container and in your regular host. Make sure that you set your PATH to include /usr/local/bin.

Now we have Meson and the ninja build system ready to go!

Building a simple sycl app with Meson

Let's start with a fresh directory. The sample sycl code is simple and easy enough to turn into a Meson-based project. Please keep in mind that this is not meant to be a complete tutorial on Meson. If you have questions, please respond to the blog post and I will do my best to answer.

$ mkdir -p $HOME/src/simple-oneapi
$ cd $HOME/src/simple-oneapi

The first thing to do is to write a meson.build file in the top level directory. The meson.build file will contain the following:

project('simple-oneapi', ['cpp', 'c'],
        version: '0.1.0',
    meson_version: '>= 0.59.0',
  default_options: [ 'warning_level=2', 'werror=false', 'cpp_std=gnu++2a', ],
)
subdir('src')

This file will define the project and its prerequisites. From here, you can see that we are defining a project that will use the C++ language version and requires a Meson version above 0.59.0. You can also define what the default compiler options are for the project. Finally, it defines that there are other sub directories with code.

For those who use make, it should be familiar to have each sub-directory have its own meson.build files to define how the code in that directory will be built. This will be no different.

$ mkdir src
$ cd src

Since we defined a src sub-directory, we're going to go ahead and create one and then add our simple oneapi source code and a meson file.

Create a file called simple-oneapi.cpp with this source code:

#include <sycl/sycl.hpp>

int main() {
  // Creating buffer of 4 ints to be used inside the kernel code
  sycl::buffer<sycl::cl_int, 1> Buffer(4);

  // Creating SYCL queue
  sycl::queue Queue;

  // Size of index space for kernel
  sycl::range<1> NumOfWorkItems{Buffer.size()};

  // Submitting command group(work) to queue
  Queue.submit([&](sycl::handler &cgh) {
    // Getting write only access to the buffer on a device
    auto Accessor = Buffer.get_access<sycl::access::mode::write>(cgh);
    // Executing kernel
    cgh.parallel_for<class FillBuffer>(
        NumOfWorkItems, [=](sycl::id<1> WIid) {
        // Fill buffer with indexes
        Accessor[WIid] = (sycl::cl_int)WIid.get(0);
        });
  });

  // Getting read only access to the buffer on the host.
  // Implicit barrier waiting for queue to complete the work.
  const auto HostAccessor = Buffer.get_access<sycl::access::mode::read>();

  // Check the results
  bool MismatchFound = false;
  for (size_t I = 0; I < Buffer.size(); ++I) {
    if (HostAccessor[I] != I) {
    std::cout << "The result is incorrect for element: " << I
                << " , expected: " << I << " , got: " << HostAccessor[I]
                << std::endl;
    MismatchFound = true;
    }
  }

  if (!MismatchFound) {
    std::cout << "The results are correct!" << std::endl;
  }

  return MismatchFound;
}

Your src directory should look like:

$ ls
simple-oneapi.cpp

Now, we are going to create a meson.build file to handle compiling simple-oneapi.cpp.

Recall that we compiled this source code using clang++ -fsycl - so we're going to have to duplicate that behavior.

Create a meson.build file with this content:

simple_oneapi_sources = files('simple-oneapi.cpp')

simple_oneapi_deps = [
]
executable('simple-oneapi', simple_oneapi_sources,
  link_args:'-fsycl',
  cpp_args:'-fsycl',
  dependencies: simple_oneapi_deps,
  install: true, install_dir: '/var/home/sri/Projects/simple-oneapi/bin'
)

This demonstrates how easy to understand the syntax of Meson is, and it contributes quite a bit to maintainability, especially if your build system gets more complex.

Define our source code

simple_oneapi_sources = files('main.cpp')

This tells Meson what source files you have. It'll be a comma delineated list of sources.

You can define more source code files like so:

simple_oneapi_sources = files('simple-oneapi.cpp', 'aux1.cpp')

Define our dependencies

simple_oneapi_deps = [
]

This will be a list of dependencies for this project. In this case, we don't have any dependencies as this is a fairly simple example.

The dependencies are generally discovered through pkg-config. If you wanted to add a dependency, it would look something like this:

simple_oneapi_deps = [
    dependency('zlib'),
    dependency('cups', method: 'pkg-config'),
]

See, the Meson documentation on dependencies for more information on dependencies.

Getting back to our example:

Defining the final compile

We now want to create a binary executable that will ultimately put our sources and the required dependencies and compile them all together.

executable('simple-oneapi', simple_oneapi_sources,
  link_args:'-fsycl',
  cpp_args:'-fsycl',
  dependencies: simple_oneapi_deps,
  install: true, install_dir: '/var/home/sri/Projects/simple-oneapi/bin'
)

We define our executable to be the sources we have defined, with the linker and pre-processor flags required. Finally, a location of where to put the resulting binary when we want to install it.

That's pretty much it. What made this tricky is that SYCL is a define-your-own-environment and so, we were not able to take advantage of a lot of the built-ins that Meson has. Instead, we had to set everything up manually. For instance, the link_args was required in order to compile the object files in the final compile.

Setting up our build system

We now have all the elements to put together our build system and getting our compile going. Let's see how we can do that.

Let's go back to the top directory of our project.

$ cd ~/src/simple-oneapi

To create this build system, we need to make sure that we set the right compiler. In this case, we are using clang++. Most of you should already be familiar with using environment variables to set up the environments.

$ CC=clang CXX=clang++ meson setup builddir

Meson does not support in-tree source code builds, so you must always define a build directory.

The result should look like this:

The Meson build system
Version: 1.0.0
Source dir: /var/home/sri/Projects/simple-oneapi
Build dir: /var/home/sri/Projects/simple-oneapi/builddir
Build type: native build
Project name: simple-oneapi
Project version: 0.1.0
C compiler for the host machine: clang (clang 16.0.0 "clang version 16.0.0 (https://github.com/intel/llvm 08be083e07b1fd6437267e26adb92f1b647d57dd)")
C linker for the host machine: clang ld.bfd 2.34
C++ compiler for the host machine: clang++ (clang 16.0.0 "clang version 16.0.0 (https://github.com/intel/llvm 08be083e07b1fd6437267e26adb92f1b647d57dd)")
C++ linker for the host machine: clang++ ld.bfd 2.34
Host machine cpu family: x86_64
Host machine cpu: x86_64
Build targets in project: 1

Found ninja-1.11.1.git.kitware.jobserver-1 at /var/home/sri/.local/bin/ninja

We are now ready to build this simple oneapi project.

To build our project, we simply do:

$ cd builddir
$ ninja

The resultant binary will be built in the src/ directory. Alternatively, if you want to be consistent especially if you're using the same codebase to build on windows and let Meson figure out which backend to use.

$ cd builddir
$ meson compile

You can find the results in the src/ directory.

$ cd src
$ ./simple-oneapi
The results are correct!

So, now we have successfully built a simple oneapi binary using Meson!!

There are several possibilities to using Meson as a build system. Meson integrates well with CMake and other build systems, so you would not have to rebuild your system from scratch.

The greatest advantage of Meson is speed and simplicity on the Linux platform.

Interested in learning more about Meson and being part of the community? Find out more at https://mesonbuild.com/. There is a Meson community on Matrix - https://matrix.to/#/#mesonbuild:matrix.org. I also highly encourage you to read Jussi Pakkane’ blog at https://nibblestew.blogspot.com/.

In the next and final blog post, we'll talk about how we can use GNOME Builder in conjunction with our container and Meson to finally put a user-friendly developer environment to write oneAPI applications on the Linux platform.

Modern Software Development Tools and oneAPI Part 1

2022-12-31T23:50:26-07:00

This will be the last blog post for this year (unless, I manage to get a second one in by the stroke of midnight tomorrow!).
I wanted to end 2022 with a departure from the last two blog posts. In this one, we're going to be looking at oneAPI toolchain from a different perspective.

I wanted to build a pure oneAPI environment that uses two things:

A different build system than your usual CMake and
an opportunity to use a Linux based IDE to write code.

Most HPC code runs on Linux in Data center. Linux is known for using arcane tools like VIM or Emacs and a build system. Sure, you can use VScode on Linux and that can be a viable option. However, I am an open-source person; I live and breathe the ideology and I am a true believer. oneAPI is an open platform, and we should use it on an open platform.

The two pieces of software I want to introduce you all is GNOME Builder and Meson.

GNOME Builder

GNOME Builder is an IDE that is primarily targeted at building GNOME-based applications. Unlike your typical IDE, GNOME Builder uses containers for building software internally. Containers in these case are flatpak SDKs - containers that contain everything you need to build an application. With GNOME Builder, you can get started writing code without the tedium of installing the entire software development toolchain you'd need to build said applications. That means, you aren't going to need to install a compiler, linker, libraries and anything else. Everything is all-inclusive - much like a vacation resort in Cancun, say! :-)

Back to Builder! oneAPI is not one of the options for building software inside GNOME Builder. GNOME Builder includes containers to build against GNOME software.

GNOME Builder is the brainchild of Christian Hergert, a long time Free Software programmer who was frustrated with the current state of tools to build software within the application community and started the Builder project about 7 years ago and was initially funded by a kickstarter. Through some great luck, Christian is now paid by Red Hat to help build GNOME Builder as well as improving the application building story on Linux and other platforms.

There is something wonderfully intriguing about Builder which is why I'm highlighting it here and why I picked it as the IDE of choice to write oneAPI-related code.

This one special trick

Builder will allow you to use a podman or docker container to run everything. So in this case, we're going to create that all-inclusive experience! Once you've done the work of building a container with all the oneAPI tools into it, others can then re-use the container as a fixed environment and other folx can clone the project you are working on and then easily collaborate.

Meson

The next piece of software I wanted to highlight is Meson. Meson is a build system that is written in python and tries to be an intuitive system that tries to do the right thing through easily understandable syntax. For those of you who have ever used autoconf - this was the application community's response to autoconf.

Autoconf during its heyday was a massive boon to those who were writing code that could work on many different UNIX and Linux distributions. However, autoconf was difficult to figure out and most folx simply copy from another project and then move on. Writing anything sophisticated required extensive, expended effort.

In my personal opinion, build systems are really hard to get right and there are so many oddities in how we build software that a build system has to get right, in order to be effective.

Meson is written by Finnish programmer, Jussi Pakkanen, who was frustrated with the current state of build systems and their arcane configuration syntax and sometimes rather unexpected behaviors!

Meson can be better described as a system that generates the configurations for build systems to use. It isn't a full-fledged build system like Make or CMake. In fact, you could easily re-use CMake configuration files in Meson. It has a concept of a backend and can generate config for Xcode on MacOS, VScode on Windows and Ninja on Linux systems. Meson easily integrates with profilers and debuggers and is designed not to build within source tree but in a designed build area.

For those not familiar with ninja, it is an extremely fast build system that has been shown to be effective in building software very quickly!!

Build an oneAPI Container

In this first part, we will focus on building an oneAPI container based on the Ubuntu 20.04 LTS release since that is what oneAPI works optimally on.

I will be using Fedora 37 SilverBlue edition. I like SilverBlue as it is built with containerized environments in mind. It allows you to build different container environments that you can enter and exit from on the command line and still easily integrate with the desktop.

Let's start then with building our oneAPI environment so that it will be able to run a simple oneAPI sample program.

Staying true to the spirit of open-source, I will build this environment from source and only use what's available on GitHub.

To start, find and install the 'distrobox' tool on your distro.

dnf install distrobox -y

Distrobox allows you to create containerized environments from the command line.

I use podman as I live in the Fedora world. Podman is a command line compatible version of Docker. It's reasonable that the following could be done through a Dockerfile but the oneAPI libraries changes often enough that this blog post would become stale in short order.

$ distrobox create oneapi -i docker.io/library/ubuntu:20.04

This will create a container called oneapi with an Ubuntu 20.04 setup.

$ distrobox enter oneapi

Will let you enter the container.

The beauty of distrobox is that you are in this container, it has mounted your home directory and you have essentially inherited your desktop system but the container is Ubuntu - and so you can use the Ubuntu distro tools to install software. Pretty neat, huh?

The first step is to build the DPC++ oneAPI compiler from source.

$ mkdir -p ~/src/sycl_workspace

You can use whatever area you want. I'm following this guide to build the compiler.

Prequisites

Let's first grab our pre-requisites for building.

$ apt install git python3 ninja gcc c++ libstdc++ libstdc++-9-dev python3-pip python3-distutils python-distutils-extra python3-psutil -y $ pip3 install meson $ pip3 install ninja

Now that we have our build environment.

Build DPC++ Compiler

$ cd ~/src/sycl_workspace $ export DPCPP_HOME=`pwd` $ git clone https://github.com/intel/llvm -b sycl

We can start the actual build:

$ python $DPCPP_HOME/llvm/buildbot/configure.py $ python $DPCPP_HOME/llvm/buildbot/compile.py

At the end of this exercise, you should have a working oneAPI DPC++ compiler.

But we aren't done yet - we still need to add some of the oneAPI libraries and runtimes to make our simple oneAPI example work.

We first need to install our low level runtimes: the things that recognizes accelerators. For now, we'll use the ones that recognize the x86 Intel processors as that is what is on my laptop right now.

We will first need to identify the latest versions of the runtimes we need to download. You need to look this up in the dependency.conf.

$ sudo mkdir -p /opt/intel 
$ sudo mkdir -p /etc/OpenCL/vendors/intel_fpgaemu.icd
$ cd /tmp
$ wget https://github.com/intel/llvm/releases/download/2022-WW50/oclcpuexp-2022.15.12.0.01_rel.tar.gz
$ wget https://github.com/intel/llvm/releases/download/2022-WW50/fpgaemu-2022.15.12.0.01_rel.tar.gz
$ sudo bash
# cd /opt/intel
# mkdir oclfpgaemu-<fpga_version>
# cd oclfpgaemu-<fpga_version>
# tar xvfpz /tmp/fpgaemu-2022.15.12.0.01_rel.tar.gz
# cd ..
# mkdir oclcpuexp_<cpu_version>
# cd oclcpuexp-<cpu_version>
# tar xvfpz /tmp/oclcpuexp-<cpu_version>
# cd ..

Now to create some configuration files.

# pwd
/opt/intel
# echo  /opt/intel/oclfpgaemu_<fpga_version>/x64/libintelocl_emu.so >
  /etc/OpenCL/vendors/intel_fpgaemu.icd
# echo /opt/intel/oclcpuexp_<cpu_version>/x64/libintelocl.so >
  /etc/OpenCL/vendors/intel_expcpu.icd

We'll need to grab a release of oneTBB from github

$ cd /tmp
$ wget https://github.com/oneapi-src/oneTBB/releases/download/v2021.7.0/oneapi-tbb-2021.7.0-lin.tgz

and now extract it.

$ cd /opt/intel
$ sudo bash
# tar xvfpz /tmp/oneapi-tbb-2021.7.0-lin.tgz

We'll need to reference some of the libraries in the oneTBB directory in our build.

# ln -s /opt/intel/oneapi-tbb-<tbb_version>/lib/intel64/gcc4.8/libtbb.so /opt/intel/oclfpgaemu_<fpga_version>/x64
# ln -s /opt/intel/oneapi-tbb-<tbb_version>/lib/intel64/gcc4.8/libtbbmalloc.so /opt/intel/oclfpgaemu_<fpga_version>/x64
# ln -s /opt/intel/oneapi-tbb-<tbb_version>/lib/intel64/gcc4.8/libtbb.so.12 /opt/intel/oclfpgaemu_<fpga_version>/x64
# ln -s /opt/intel/oneapi-tbb-<tbb_version>/lib/intel64/gcc4.8/libtbbmalloc.so.2 /opt/intel/oclfpgaemu_<fpga_version>/x64

# ln -s /opt/intel/oneapi-tbb-<tbb_version>/lib/intel64/gcc4.8/libtbb.so /opt/intel/oclcpuexp_<cpu_version>/x64
# ln -s /opt/intel/oneapi-tbb-<tbb_version>/lib/intel64/gcc4.8/libtbbmalloc.so /opt/intel/oclcpuexp_<cpu_version>/x64
# ln -s /opt/intel/oneapi-tbb-<tbb_version>/lib/intel64/gcc4.8/libtbb.so.12 /opt/intel/oclcpuexp_<cpu_version>/x64
# ln -s /opt/intel/oneapi-tbb-<tbb_version>/lib/intel64/gcc4.8/libtbbmalloc.so.2 /opt/intel/oclcpuexp_<cpu_version>/x64

Now we need to configure the library paths:

# echo /opt/intel/oclfpgaemu_<fpga_version>/x64 > /etc/ld.so.conf.d/libintelopenclexp.conf
# echo /opt/intel/oclcpuexp_<cpu_version>/x64 >> /etc/ld.so.conf.d/libintelopenclexp.conf
# ldconfig -f /etc/ld.so.conf.d/libintelopenclexp.conf

and we're done! Now we need to make sure that this toolchain actually works. So run this test.

Make sure you are not root.

$ python $DPCPP_HOME/llvm/buildbot/check.py

If you come back with no failure then, congratulations, you're in good shape!! Sometimes, there might be a few missing dependencies, especially when it comes to python.

We are now ready to create a simple SYCL application and test. I'm going to re-use the one that is located on Github.

Let's create our workspace and build this sample project.

$ mkdir -p ~/src/simple-oneapi/
$ cd ~/src/simple-oneapi
$ export PATH=$DPCPP_HOME/llvm/build/bin:$PATH
$ export LD_LIBRARY_PATH=$DPCPP_HOME/llvm/build/lib:$LD_LIBRARY_PATH
$ cat > simple-oneapi.cpp

#include <sycl/sycl.hpp>

int main() {
  // Creating buffer of 4 ints to be used inside the kernel code
  sycl::buffer<sycl::cl_int, 1> Buffer(4);

  // Creating SYCL queue
  sycl::queue Queue;

  // Size of index space for kernel
  sycl::range<1> NumOfWorkItems{Buffer.size()};

  // Submitting command group(work) to queue
  Queue.submit([&](sycl::handler &cgh) {
    // Getting write only access to the buffer on a device
    auto Accessor = Buffer.get_access<sycl::access::mode::write>(cgh);
    // Executing kernel
    cgh.parallel_for<class FillBuffer>(
        NumOfWorkItems, [=](sycl::id<1> WIid) {
          // Fill buffer with indexes
          Accessor[WIid] = (sycl::cl_int)WIid.get(0);
        });
  });

  // Getting read only access to the buffer on the host.
  // Implicit barrier waiting for queue to complete the work.
  const auto HostAccessor = Buffer.get_access<sycl::access::mode::read>();

  // Check the results
  bool MismatchFound = false;
  for (size_t I = 0; I < Buffer.size(); ++I) {
    if (HostAccessor[I] != I) {
      std::cout << "The result is incorrect for element: " << I
                << " , expected: " << I << " , got: " << HostAccessor[I]
                << std::endl;
      MismatchFound = true;
    }
  }

  if (!MismatchFound) {
    std::cout << "The results are correct!" << std::endl;
  }

  return MismatchFound;
}

Let's build our simple oneapi source code!

$ clang++ -fsycl simple-sycl-app.cpp -o simple-sycl-app

It should compile and run without any errors.

If all works as anticipated, you should have a working setup.

Setting up the container to code SYCL when you enter

Now, the next step is to make this container useful when you enter and have it always ready to build a sycl app.

Exit out of the container using the 'exit' command and you should be back on the host operating system.

Type:
$ uname -a

On my system, I get:

Linux fedora 6.0.13-300.fc37.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Dec 14 16:15:19 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Re-enter the container:

$ distrobox enter oneapi
$ uname -a
Linux oneapi.fedora 6.0.13-300.fc37.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Dec 14 16:15:19 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

You will notice that after "Linux" when you are in the container, there is a "oneapi" prefix to fedora.

We can take advantage of that. Let's make sure that when we enter the container that we can set things up from the shell perspective to be ready to write SYCL code.

Add this bit to your .bashrc:

oneapi=``uname -a | grep -c oneapi``

if [ $oneapi -gt 0 ]; then
   echo "Initializing oneAPI"
   export DPCPP_HOME="/var/home/sri/src/dpcplusplus"
   export PATH="$PATH:/var/home/sri/.local/bin:$DPCPP_HOME/llvm/build/bin"
   export LD_LIBRARY_PATH="$DPCPP_HOME/llvm/build/lib"
fi

Make sure you replace 'sri' with your your login details

Now when we enter the 'oneapi' container our environment will be properly initialized.

Let's stop here, and we'll pick it up in the next post. The next post will focus on creating a meson setup around this simple oneapi code. Part 3 will focus on taking our meson configured source code and using GNOME Builder to build it. Stay tuned!

Photo by iMattSmart on Unsplash

Community Map

2022-11-23T00:00:00-07:00

[{‘type’: ‘text/html’, ‘language’: ‘en’, ‘base’: ‘https://hpc-social.github.io/news/2022/community-map/’, ‘value’: ‘After an evening coding session, we are happy to announce that the community map\n (and associated automation) is live!\nThis post will capture the screenshot of the first set of entries, which is what\nwe wanted to write about. This was only announced days ago (and we only have a few entries)\nbut we are taken aback by the (so far) geographic diversity of the HPC community!\nSo far with a small number of entries, we are hitting the community in Europe,\nthe United States, and even South America. This is awesome! \n\nFinally, we’ve added a new field for a group URL to the map form page ! \nIf you’ve already provided your group don’t worry - we looked it up for you, but future groups will\nbe able to provide their group website URLs to automatically appear on the map\nalongside the popup. What fun would it be to browse groups without being able\nto look closer? 🤔️ \n\nWe have more to come, but this is a quick update before the break. Happy\nholiday break to our HPC community, whether you eat turkey or not, do something else,\nor are just using it as a glorious time to rest or do fun things. 🎉️ ‘}]

Welcome to HPC.social!

2022-11-20T00:00:00-07:00

[{‘type’: ‘text/html’, ‘language’: ‘en’, ‘base’: ‘https://hpc-social.github.io/news/2022/welcome/’, ‘value’: ‘We are excited to announce the launch of our community, where we hope to connect\nHPC practitioners and friends alike, and provide useful resources for the community. ‘}]

How the SYCL spec gets updated

2022-11-18T21:59:08-07:00

What is SYCL?

From the KHRONOS website (https://www.khronos.org/sycl/)

SYCL (pronounced 'sickle') is a royalty-free, cross-platform abstraction layer that enables code for heterogeneous processors to be written using standard ISO C++ with the host and kernel code for an application contained in the same source file.

In short, SYCL is a C++ extension offered as an industry replacement to CUDA. SYCL allows you to run on many kinds of accelerators like FPGAs, GPUs, and CPUs'- you could scale it from a Raspberry Pi to the latest GPUs and CPUIs on the market.

We will not go too much into SYCL from a code perspective. That will be the focus of future blog posts. In this post, we focus on how SYCL gets updated as an industry standard.

KHRONOS

The caretaker of the SYCL spec is Khronos a non-profit standards body that is home to several projects focused on graphics, machine learning, parallel computing, VR and visual computing. You will note that all these pieces work together to build frameworks. Some of the more visible ones that you might be familiar with are openGL and Vulkan.

Khronos consists of members who represent industry players from companies, non-profits, and individuals. Every project will have a set of members.

How a feature becomes part of the spec

Let us start with how a feature gets introduced into the spec. Usually the process of adding a feature starts when a member company identifies a problem or an idea internally through feedback, bug reports, or strategic direction.

Once that feature is fleshed out, it is introduced to the SYCL working group. The working group then discusses the proposal as to how the feature evolves, and the proposal gets iterated on until people are satisfied with the changes. The feature is then integrated into the spec.

Once the feature is in the SYCL spec, the working group will ratify the feature with a vote. The vote is important because it means that the output of that feature is now under the protection of the Khronos group, which means that if another member objects because of an IP issue later they will not have grounds to do so, and the implementation of the feature is protected.

What happens before a member goes to the working group?

Most of the time, the member company already implemented the feature they were looking to integrate into the spec to ensure that the desired feature solves the problem at hand. This also assures the working group that the feature can be implemented.

As a specific example, Intel, when they want to design a feature to extend the SYCL spec, they start a branch in the DPG++ GitHub area and then work on it publicly. Once the implementation is complete, it is considered ready for submission to the working group.

What happens after approval?

After the approval of the preliminary draft of the new spec which contains the new features, the draft is open for public comments. Usually, there is a 30-45 -day period where Khronos members can inspect the new version of the spec, file IP claims against it, or offer changes. An iterative process happens when public feedback is used to make more changes and clean up. Finally, the new changes are ratified, and the feature implementation is put into the spec.

Keep in mind that this is all happening in parallel as other member projects are also targeting the same spec with their own features.

Getting the final stamp of approval

Once you have finished ratifying the spec and produced a working implementation of the feature you will need to do a performance run using an included conformance suite provided by Khronos. Once that passes, you will be able to put the Khronos logo on your implementation of the updated spec.

That is a summary of how SYCL is updated through a standards body like Khronos.

Summary

I hope this gives you a birds eye view of the process of how the SYCL spec is updated. If you have questions - please ask in the comments! If anyone is interested in Khronos membership - happy to direct.

Supercomputing 2022

2022-11-13T02:47:14-07:00

OpenMP was in Dallas for Supercomputing 2022. We will have four tutorials, a BOF, and more.

The post Supercomputing 2022 appeared first on OpenMP.

OpenMP ARB Releases Technical Report 11

2022-11-10T18:20:32-07:00

The OpenMP® Architecture Review Board (ARB) has released Technical Report 11, the first preview of the future version 6.0 of the OpenMP API and version 5.2.1 of the collection of programming examples.

The post OpenMP ARB Releases Technical Report 11 appeared first on OpenMP.

Understanding the CaRCC Facings

2022-10-03T18:30:38-06:00

One of the biggest challenges we face as an organization supporting the research computing and data (RCD) profession is defining what it means to be an RCD professional. Although it’s clear our profession is growing and very much in-demand, the lines between RCD and other types of jobs are often blurred, and our professional titles rarely reflect the breadth of what we do. Supporting the needs of researchers is distinct and often very different from providing enterprise technology services to a general user population (even though many of us do both and overlap certainly exists). The Campus Research Computing Consortium (CaRCC) focuses on the challenges and opportunities unique to providing computational, data, and other related services to researchers.

Am I an RCD professional?

Most of us would agree that the people who manage university supercomputers are RCD professionals, but am I an RCD professional if I help researchers navigate data and my title is Librarian or Data Scientist? Am I an RCD professional if I’m a programmer who develops software that supports a research project? Am I an RCD professional if I’m not involved in hands-on technical work but do outreach and education to help researchers find and use technology and data resources?

If you are supporting researchers in any way through systems, software, data, security, networking, data center operations, or education and consulting, yes –you are an RCD professional!

Okay, so I’m an RCD professional. How do I find “my people” within this broad and diverse profession?

We segment the RCD profession into five (non-mutually exclusive) focus areas that we call “Facings” – as in “this is what/whom I’m facing much of the time.” Most of us do things related to more than one of the Facings.

What or whom are you facing?

Use these descriptions to help you find your own RCD peer groups or to better understand the Facings.

Researcher-Facing RCD Professionals – We work directly with researchers in areas such as outreach, education, training, consulting, facilitation, liaising, etc.

Systems-Facing RCD Professionals – We work with systems – including systems administration, planning, engineering, networking, optimization, systems and network security, instruments and sensors, middleware, and the like.

Data-Facing RCD Professionals – We are responsible for data discovery, curation, management, publishing/sharing, data science, data security, and data-related functions.

Software-Facing RCD Professionals – We are involved in software development, engineering portability, installation, optimization, support, and related activities.

Strategy and Policy-Facing RCD Professionals – We help set organizational policies, develop programs and personnel management strategies, lead service evaluations, and work in areas related to funding, finances, security policy, or other strategic initiatives.

I get it! Now what?

Once you’ve found your Facings, join the People Network and sign up to join your Facings’ meetings and discussions. CaRCC is an open and inclusive community; we actively encourage and welcome new members!

IWOMP 2022- International Workshop on OpenMP

2022-08-09T22:34:26-06:00

September 27 - 30
at the University of Tennessee at Chattanooga, USA,

The post IWOMP 2022: International Workshop on OpenMP appeared first on OpenMP.

ECP SOLLVE organizes a bi-weekly session for assistance with OpenMP technical problems

2022-06-17T19:04:39-06:00

Every two weeks | Teleconference
Dedicated office hours every two weeks from 10:30 a.m. to 11:30 a.m. central time

The post ECP SOLLVE organizes a bi-weekly session for assistance with OpenMP technical problems appeared first on OpenMP.

2021 RCD Capabilities Model Community Data report now available

2022-05-25T17:49:57-06:00

The second Research Computing and Data (RCD) Capabilities Model Community Dataset report is now available through CaRCC’s RCD Nexus as part of a project funded by the National Science Foundation. The report aggregates assessments contributed by 51 higher education institutions, providing insight into the current state of support for RCD.

Report data is intended to support strategic planning and decision-making for campus leadership, funding agencies, RCD leaders, and others interested in advanced technologies that enable research.

The RCD Capabilities Model Community Dataset report is now available.

Themes that emerged from the 2021 data include:

There is generally stronger support for Researcher-Facing, System-Facing, and Strategy and Policy-Facing areas, than for Data-Facing and Software-Facing capabilities.
R1 institutions have much higher levels of coverage than other Carnegie Classifications, particularly in certain areas.
EPSCoR institutions have significant gaps in capabilities coverage relative to institutions in other states, including dramatic gaps in certain areas of Data-Facing support.

“This level of data has never before been readily available to the RCD community,” said Patrick Schmitz, co-principal investigator on the RCD Nexus project. “We were intrigued to see that institutions who completed a second assessment in 2021 saw a marked increase in their capabilities related to RCD strategy and policy. It’s possible the assessment is already helping them improve their programs. We will work to continue expanding the dataset in the coming years, and are pleased to offer the RCD community a solid baseline dataset to help inform strategic planning.”

This material is based upon work supported by the National Science Foundation under Grant OAC-2100003. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

OpenMP BOF at ECP Community Days

2022-05-11T21:04:13-06:00

The video recording of the OpenMP BOF at the Exascale Computing Project community event on May 11 is now available.

The post OpenMP BOF at ECP Community Days appeared first on OpenMP.

OpenMP Offload in Applications of the Exascale Computing Project

2022-05-06T15:54:38-06:00

This blog post describes the strategy used to implement the OpenMP offload functionality in these DoE projects.

The post OpenMP Offload in Applications of the Exascale Computing Project appeared first on OpenMP.

OpenMP ARB Appoints Jack Dongarra to its Board of Directors

2022-04-28T20:18:57-06:00

The OpenMP ARB welcomes Jack Dongarra to its Board of Directors. His insights and guidance will greatly benefit the Board.

The post OpenMP ARB Appoints Jack Dongarra to its Board of Directors appeared first on OpenMP.

Webinar- Large-Scale Materials Science Codes Porting Strategies on GPU Architectures, the BerkeleyGW Case Study

2022-04-20T05:15:48-06:00

This webinar discusses our experiences porting BerkeleyGW to three different GPU programming models (CUDA, OpenACC, and OpenMP Target) with various GPU vendor architectures.

The post Webinar: Large-Scale Materials Science Codes Porting Strategies on GPU Architectures, the BerkeleyGW Case Study appeared first on OpenMP.

ECP OpenMP Virtual Hackathon 2022

2022-04-08T06:04:45-06:00

April 8-13. Join us for this hackathon where the goal is to port, optimize and evolve applications towards the latest OpenMP versions

The post ECP OpenMP Virtual Hackathon 2022 appeared first on OpenMP.

Webinar- Using OpenMP to Harness GPUs for Core-Collapse Supernova Simulations with GenASiS

2022-02-18T01:14:49-07:00

Join us for a webinar introducing how to use OpenMP to harness the awesome computational power of GPUs with GenASiS for scientific simulations of core-collapse supernovas.

The post Webinar: Using OpenMP to Harness GPUs for Core-Collapse Supernova Simulations with GenASiS appeared first on OpenMP.

hpc.social - Community Syndicated Blog

New Board Members van der Pas and Stotzer

OpenMP Example Book Survey

New Python Subcommittee, and Anaconda joins OpenMP ARB

IWOMP 2026

OpenMP ARB Releases Technical Report 14

Supercomputing 2025

IWOMP 2025

NextSilicon joins the OpenMP

ISC 2025

Status of OpenMP API 6.0 Implementations

IWOMP 2025

OpenMP® ARB Releases OpenMP 6.0 for Easier Programming

Supercomputing 2024

OpenMP ARB Releases Public Comment Draft of OpenMP 6.0

Asynchronous GPU Programming in OpenMP

The OpenMP ARB welcomes new member CEA

IWOMP 2024

Looking forward to 2024!

Looking forward to next year!

Goals for next year

Have a wonderful holiday season and a Happy New Year!

Building oneAPI from Source

Build oneAPI completely from git

Setting up the build environment

Install the appropriate packages

Understanding the oneAPI Build

Intel Graphics Engine

ocl-icd

Install GMMLib

Install NEO

Install level-zero

Install the DPC++ Compiler

Install oneTBB

Set the LD_LIBRARY_PATH

Test the environment

Supercomputing 2023 - HPC Social Summary!

HPC Social Virtual Noodles Award

Beowulf Bash

I Want to Run my MPI

Official Greeting

Texas Tech Tiny Cluster!

The HPC Social Community

Supercomputing 2023

OpenMP ARB Releases Technical Report 12

Fortran Package Manager and OpenMP

oneAPI moves to Unified Acceleration Foundation (UXL)

IWOMP 2023

Migrating reduction operations to SYCL in a molecular docking application

Integer Reductions to find the number of evaluations

Finding the minimum energy

Summary

IWOMP 2023

Supercomputing 2023

Awesome oneAPI announced!

The hpc.social project - talk at EasyBuild Users Meeting

OpenMP® ARB adds new member Samsung

Using oneAPI AI Toolkits from Intel and Accenture Part 2

Introduction

Personalized Retail Experiences with Enhanced Customer Segmentation

Faster session Notes with Speech-to-Text AI for Healthcare Providers

Summary

Call to Action

Training AI with oneAPI part 1

oneAPI and AI

Accenture and Intel

oneAPI AI Toolkits

Disease Prediction

Increase Mortgage Loan Default Risk Prediction Speed

Call to Action

SiFive joins the OpenMP® effort

Embedded Systems and the OpenMP® API

Modern Software Development Tools and oneAPI Part 3

Creating a Project

Setup the build system

Set up our source

Set the build and run environment to our SYCL container.

2023 ECP COMMUNITY BOF DAYS

Modern Software Development Tools and oneAPI Part 2

Modern Software Development Tools and oneAPI Part 2