Below the vendor-specific APIs are the vendor specific compilers. As far as modern machine learning practitioners go, these compilers are very rarely interacted with directly. As for our own representation of the ML stack, these compilers are the lowest level building blocks that we consider. Of course, we could talk about assembly languages and byte code, but this is outside the scope of what is really relevant for ML practitioners when considering their software stack.
Intel C++ Compiler Classic (ICC) is the first of Intel’s two C, C++, SYCL, and Data Parallel C++ (DPC++) compilers for Intel processor-based systems. It is available for Windows, Linux, and macOS operating systems. It targets general-purpose Intel x86-64 architecture CPUs.
Intel oneAPI DPC++/C++ Compiler (ICX) is the second of Intel’s two C, C++, SYCL, and Data Parallel C++ (DPC++) compilers for Intel processor-based systems. Again, it is available for Windows, Linux, and macOS operating systems. Unlike ICC, It generates code for both Intel’s general-purpose x86-64 CPUs and also GPUs. Specifically, it targets Intel IA-32, Intel 64 (aka x86-64), Core, Xeon, and Xeon Scalable processors, as well as GPUs including Intel Processor Graphics Gen9 and above, Intel Xe architecture, and Intel Programmable Acceleration Card with Intel Arria 10 GX FPGA. It builds on the SYCL specification from The Khronos Group. It is designed to allow developers to reuse code across hardware targets (CPUs and accelerators such as GPUs and FPGAs) and perform custom tuning for a specific accelerator. ICX adopts LLVM for faster build times, and benefits from supporting the latest C++ standards.
The Nvidia CUDA Compiler (NVCC) is a proprietary compiler by NVIDIA intended for use with CUDA. CUDA code runs on both the CPU and GPU. NVCC separates these two parts and sends host code (the part of code which will be run on the CPU) to a C compiler like GCC or Intel C++ Compiler Classic (ICC) or Microsoft Visual C++ Compiler, and sends the device code (the part which will run on the GPU) to the GPU. The device code is further compiled by NVCC. Like ICX, NVCC is also based on LLVM.