[Clang][SYCL] Introduce clang-sycl-linker to link SYCL offloading device code (Part 1 of many) (#112245)

This PR is one of the many PRs in the SYCL upstreaming effort focusing on device code linking during the SYCL offload compilation process. RFC: https://discourse.llvm.org/t/rfc-offloading-design-for-sycl-offload-kind-and-spir-targets/74088 In this PR, we introduce a new tool that will be used to perform device code linking for SYCL offload kind. It accepts SYCL device objects in LLVM IR bitcode format and will generate a fully linked device object that can then be wrapped and linked into the host object. A primary use case for this tool is to perform device code linking for objects with SYCL offload kind inside the clang-linker-wrapper. It can also be invoked via clang driver as follows: `clang --target=spirv64 --sycl-link input.bc` Device code linking for SYCL offloading kind has a number of known quirks that makes it difficult to use in a unified offloading setting. Two of the primary issues are: 1. Several finalization steps are required to be run on the fully-linked LLVM IR bitcode to gaurantee conformance to SYCL standards. This step is unique to SYCL offloading compilation flow. 2. SPIR-V LLVM Translator tool is an extenal tool and hence SPIR-V IR code generation cannot be done as part of LTO. This limitation will be lifted once SPIR-V backend is available as a viable LLVM backend. Hence, we introduce this new tool to provide a clean wrapper to perform SYCL device linking. Co-Author: Michael Toguchi Thanks --------- Signed-off-by: Arvind Sudarsanam <arvind.sudarsanam@intel.com>
2024-10-31 07:39:55 -07:00
parent c485ee1968
commit eeee5a44bb
14 changed files with 755 additions and 3 deletions
--- a/clang/docs/ClangSYCLLinker.rst
+++ b/clang/docs/ClangSYCLLinker.rst
@@ -0,0 +1,82 @@
+=======================
+Clang SYCL Linker
+=======================
+
+.. contents::
+   :local:
+
+.. _clang-sycl-linker:
+
+Introduction
+============
+
+This tool works as a wrapper around the SYCL device code linking process.
+The purpose of this tool is to provide an interface to link SYCL device bitcode
+in LLVM IR format, SYCL device bitcode in SPIR-V IR format, and native binary
+objects, and then use the SPIR-V LLVM Translator tool on fully linked device
+objects to produce the final output.
+After the linking stage, the fully linked device code in LLVM IR format may
+undergo several SYCL-specific finalization steps before the SPIR-V code
+generation step.
+The tool will also support the Ahead-Of-Time (AOT) compilation flow. AOT
+compilation is the process of invoking the back-end at compile time to produce
+the final binary, as opposed to just-in-time (JIT) compilation when final code
+generation is deferred until application runtime.
+
+Device code linking for SYCL offloading has several known quirks that
+make it difficult to use in a unified offloading setting. Two of the primary
+issues are:
+1. Several finalization steps are required to be run on the fully linked LLVM
+IR bitcode to guarantee conformance to SYCL standards. This step is unique to
+the SYCL offloading compilation flow.
+2. The SPIR-V LLVM Translator tool is an external tool and hence SPIR-V IR code
+generation cannot be done as part of LTO. This limitation can be lifted once
+the SPIR-V backend is available as a viable LLVM backend.
+
+This tool has been proposed to work around these issues.
+
+Usage
+=====
+
+This tool can be used with the following options. Several of these options will
+be passed down to downstream tools like 'llvm-link', 'llvm-spirv', etc.
+
+.. code-block:: console
+
+  OVERVIEW: A utility that wraps around the SYCL device code linking process.
+  This enables linking and code generation for SPIR-V JIT targets and AOT
+  targets.
+
+  USAGE: clang-sycl-linker [options]
+
+  OPTIONS:
+    --arch <value>                Specify the name of the target architecture.
+    --dry-run                     Print generated commands without running.
+    -g                            Specify that this was a debug compile.
+    -help-hidden                  Display all available options
+    -help                         Display available options (--help-hidden for more)
+    --library-path=<dir>          Set the library path for SYCL device libraries
+    --device-libs=<value>         A comma separated list of device libraries that are linked during the device link
+    -o <path>                     Path to file to write output
+    --save-temps                  Save intermediate results
+    --triple <value>              Specify the target triple.
+    --version                     Display the version number and exit
+    -v                            Print verbose information
+    -spirv-dump-device-code=<dir> Directory to dump SPIR-V IR code into
+    -is-windows-msvc-env          Specify if we are compiling under windows environment
+    -llvm-spirv-options=<value>   Pass options to llvm-spirv tool
+    --llvm-spirv-path=<dir>       Set the system llvm-spirv path
+
+Example
+=======
+
+This tool is intended to be invoked when targeting any of the target offloading
+toolchains. When the --sycl-link option is passed to the clang driver, the
+driver will invoke the linking job of the target offloading toolchain, which in
+turn will invoke this tool. This tool can be used to create one or more fully
+linked device images that are ready to be wrapped and linked with host code to
+generate the final executable.
+
+.. code-block:: console
+
+  clang-sycl-linker --triple spirv64 --arch native input.bc
--- a/clang/docs/index.rst
+++ b/clang/docs/index.rst
@@ -98,6 +98,7 @@ Using Clang Tools
   ClangOffloadBundler
   ClangOffloadPackager
   ClangRepl
+   ClangSYCLLinker

 Design Documents
 ================
--- a/clang/include/clang/Driver/Options.td
+++ b/clang/include/clang/Driver/Options.td
@@ -6775,7 +6775,10 @@ def fsycl : Flag<["-"], "fsycl">,
 def fno_sycl : Flag<["-"], "fno-sycl">,
  Visibility<[ClangOption, CLOption]>,
  Group<sycl_Group>, HelpText<"Disables SYCL kernels compilation for device">;
-
+def sycl_link : Flag<["--"], "sycl-link">, Flags<[HelpHidden]>,
+  Visibility<[ClangOption, CLOption]>,
+  Group<sycl_Group>, HelpText<"Perform link through clang-sycl-linker via the target "
+  "offloading toolchain.">;
 // OS-specific options
 let Flags = [TargetSpecific] in {
 defm android_pad_segment : BooleanFFlag<"android-pad-segment">, Group<f_Group>;
--- a/clang/lib/Driver/Driver.cpp
+++ b/clang/lib/Driver/Driver.cpp
@@ -4791,6 +4791,11 @@ Action *Driver::ConstructPhaseAction(
  if (Phase == phases::Assemble && Input->getType() != types::TY_PP_Asm)
    return Input;

+  // Use of --sycl-link will only allow for the link phase to occur. This is
+  // for all input files.
+  if (Args.hasArg(options::OPT_sycl_link) && Phase != phases::Link)
+    return Input;
+
  // Build the appropriate action.
  switch (Phase) {
  case phases::Link:
--- a/clang/lib/Driver/ToolChains/SPIRV.cpp
+++ b/clang/lib/Driver/ToolChains/SPIRV.cpp
@@ -95,7 +95,21 @@ void SPIRV::Linker::ConstructJob(Compilation &C, const JobAction &JA,
  CmdArgs.push_back("-o");
  CmdArgs.push_back(Output.getFilename());

+  // Use of --sycl-link will call the clang-sycl-linker instead of
+  // the default linker (spirv-link).
+  if (Args.hasArg(options::OPT_sycl_link))
+    Linker = ToolChain.GetProgramPath("clang-sycl-linker");
  C.addCommand(std::make_unique<Command>(JA, *this, ResponseFileSupport::None(),
                                         Args.MakeArgString(Linker), CmdArgs,
                                         Inputs, Output));
 }
+
+SPIRVToolChain::SPIRVToolChain(const Driver &D, const llvm::Triple &Triple,
+                               const ArgList &Args)
+    : ToolChain(D, Triple, Args) {
+  // TODO: Revisit need/use of --sycl-link option once SYCL toolchain is
+  // available and SYCL linking support is moved there.
+  NativeLLVMSupport = Args.hasArg(options::OPT_sycl_link);
+}
+
+bool SPIRVToolChain::HasNativeLLVMSupport() const { return NativeLLVMSupport; }
--- a/clang/lib/Driver/ToolChains/SPIRV.h
+++ b/clang/lib/Driver/ToolChains/SPIRV.h
@@ -57,8 +57,7 @@ class LLVM_LIBRARY_VISIBILITY SPIRVToolChain final : public ToolChain {

 public:
  SPIRVToolChain(const Driver &D, const llvm::Triple &Triple,
-                 const llvm::opt::ArgList &Args)
-      : ToolChain(D, Triple, Args) {}
+                 const llvm::opt::ArgList &Args);

  bool useIntegratedAs() const override { return true; }

@@ -72,6 +71,7 @@ public:
  }
  bool isPICDefaultForced() const override { return false; }
  bool SupportsProfiling() const override { return false; }
+  bool HasNativeLLVMSupport() const override;

  clang::driver::Tool *SelectTool(const JobAction &JA) const override;

@@ -81,6 +81,7 @@ protected:

 private:
  clang::driver::Tool *getTranslator() const;
+  bool NativeLLVMSupport;
 };

 } // namespace toolchains
--- a/clang/test/CMakeLists.txt
+++ b/clang/test/CMakeLists.txt
@@ -80,6 +80,7 @@ list(APPEND CLANG_TEST_DEPS
  clang-nvlink-wrapper
  clang-offload-bundler
  clang-offload-packager
+  clang-sycl-linker
  diagtool
  hmaptool
  )
--- a/clang/test/Driver/clang-sycl-linker-test.cpp
+++ b/clang/test/Driver/clang-sycl-linker-test.cpp
@@ -0,0 +1,48 @@
+// Tests the clang-sycl-linker tool.
+//
+// Test a simple case without arguments.
+// RUN: %clangxx -emit-llvm -c %s -o %t_1.bc
+// RUN: %clangxx -emit-llvm -c %s -o %t_2.bc
+// RUN: clang-sycl-linker --dry-run -triple spirv64 %t_1.bc %t_2.bc -o a.spv 2>&1 \
+// RUN:   | FileCheck %s --check-prefix=SIMPLE
+// SIMPLE: "{{.*}}llvm-link{{.*}}" {{.*}}.bc {{.*}}.bc -o [[FIRSTLLVMLINKOUT:.*]].bc --suppress-warnings
+// SIMPLE-NEXT: "{{.*}}llvm-spirv{{.*}}" {{.*}}-o a.spv [[FIRSTLLVMLINKOUT]].bc
+//
+// Test that llvm-link is not called when only one input is present.
+// RUN: clang-sycl-linker --dry-run -triple spirv64 %t_1.bc -o a.spv 2>&1 \
+// RUN:   | FileCheck %s --check-prefix=SIMPLE-NO-LINK
+// SIMPLE-NO-LINK: "{{.*}}llvm-spirv{{.*}}" {{.*}}-o a.spv {{.*}}.bc
+//
+// Test a simple case with device library files specified.
+// RUN: touch %T/lib1.bc
+// RUN: touch %T/lib2.bc
+// RUN: clang-sycl-linker --dry-run -triple spirv64 %t_1.bc %t_2.bc --library-path=%T --device-libs=lib1.bc,lib2.bc -o a.spv 2>&1 \
+// RUN:   | FileCheck %s --check-prefix=DEVLIBS
+// DEVLIBS: "{{.*}}llvm-link{{.*}}" {{.*}}.bc {{.*}}.bc -o [[FIRSTLLVMLINKOUT:.*]].bc --suppress-warnings
+// DEVLIBS-NEXT: "{{.*}}llvm-link{{.*}}" -only-needed [[FIRSTLLVMLINKOUT]].bc {{.*}}lib1.bc {{.*}}lib2.bc -o [[SECONDLLVMLINKOUT:.*]].bc --suppress-warnings
+// DEVLIBS-NEXT: "{{.*}}llvm-spirv{{.*}}" {{.*}}-o a.spv [[SECONDLLVMLINKOUT]].bc
+//
+// Test a simple case with .o (fat object) as input.
+// TODO: Remove this test once fat object support is added.
+// RUN: %clangxx -c %s -o %t.o
+// RUN: not clang-sycl-linker --dry-run -triple spirv64 %t.o -o a.spv 2>&1 \
+// RUN:   | FileCheck %s --check-prefix=FILETYPEERROR
+// FILETYPEERROR: Unsupported file type
+//
+// Test to see if device library related errors are emitted.
+// RUN: not clang-sycl-linker --dry-run -triple spirv64 %t_1.bc %t_2.bc --library-path=%T --device-libs= -o a.spv 2>&1 \
+// RUN:   | FileCheck %s --check-prefix=DEVLIBSERR1
+// DEVLIBSERR1: Number of device library files cannot be zero
+// RUN: not clang-sycl-linker --dry-run -triple spirv64 %t_1.bc %t_2.bc --library-path=%T --device-libs=lib1.bc,lib2.bc,lib3.bc -o a.spv 2>&1 \
+// RUN:   | FileCheck %s --check-prefix=DEVLIBSERR2
+// DEVLIBSERR2: '{{.*}}lib3.bc' SYCL device library file is not found
+//
+// Test if correct set of llvm-spirv options are emitted for windows environment.
+// RUN: clang-sycl-linker --dry-run -triple spirv64 --is-windows-msvc-env %t_1.bc %t_2.bc -o a.spv 2>&1 \
+// RUN:   | FileCheck %s --check-prefix=LLVMOPTSWIN
+// LLVMOPTSWIN: -spirv-debug-info-version=ocl-100 -spirv-allow-extra-diexpressions -spirv-allow-unknown-intrinsics=llvm.genx. -spirv-ext=
+//
+// Test if correct set of llvm-spirv options are emitted for linux environment.
+// RUN: clang-sycl-linker --dry-run -triple spirv64  %t_1.bc %t_2.bc -o a.spv 2>&1 \
+// RUN:   | FileCheck %s --check-prefix=LLVMOPTSLIN
+// LLVMOPTSLIN: -spirv-debug-info-version=nonsemantic-shader-200 -spirv-allow-unknown-intrinsics=llvm.genx. -spirv-ext=
--- a/clang/test/Driver/sycl-link-spirv-target.cpp
+++ b/clang/test/Driver/sycl-link-spirv-target.cpp
@@ -0,0 +1,9 @@
+// Tests the driver when linking LLVM IR bitcode files and targeting SPIR-V
+// architecture.
+//
+// Test that -Xlinker options are being passed to clang-sycl-linker.
+// RUN: touch %t.bc
+// RUN: %clangxx -### --target=spirv64 --sycl-link -Xlinker --llvm-spirv-path=/tmp \
+// RUN:   -Xlinker --library-path=/tmp -Xlinker --device-libs=lib1.bc,lib2.bc %t.bc 2>&1 \
+// RUN:   | FileCheck %s -check-prefix=XLINKEROPTS
+// XLINKEROPTS: "{{.*}}clang-sycl-linker{{.*}}" "--llvm-spirv-path=/tmp" "--library-path=/tmp" "--device-libs=lib1.bc,lib2.bc" "{{.*}}.bc" "-o" "a.out"
--- a/clang/test/lit.cfg.py
+++ b/clang/test/lit.cfg.py
@@ -96,6 +96,7 @@ tools = [
    "yaml2obj",
    "clang-linker-wrapper",
    "clang-nvlink-wrapper",
+    "clang-sycl-linker",
    "llvm-lto",
    "llvm-lto2",
    "llvm-profdata",
--- a/clang/tools/CMakeLists.txt
+++ b/clang/tools/CMakeLists.txt
@@ -12,6 +12,7 @@ add_clang_subdirectory(clang-nvlink-wrapper)
 add_clang_subdirectory(clang-offload-packager)
 add_clang_subdirectory(clang-offload-bundler)
 add_clang_subdirectory(clang-scan-deps)
+add_clang_subdirectory(clang-sycl-linker)
 add_clang_subdirectory(clang-installapi)
 if(HAVE_CLANG_REPL_SUPPORT)
  add_clang_subdirectory(clang-repl)
--- a/clang/tools/clang-sycl-linker/CMakeLists.txt
+++ b/clang/tools/clang-sycl-linker/CMakeLists.txt
@@ -0,0 +1,28 @@
+set(LLVM_LINK_COMPONENTS
+  ${LLVM_TARGETS_TO_BUILD}
+  Option
+  )
+
+set(LLVM_TARGET_DEFINITIONS SYCLLinkOpts.td)
+tablegen(LLVM SYCLLinkOpts.inc -gen-opt-parser-defs)
+add_public_tablegen_target(SYCLLinkerOpts)
+
+if(NOT CLANG_BUILT_STANDALONE)
+  set(tablegen_deps intrinsics_gen SYCLLinkerOpts)
+endif()
+
+add_clang_tool(clang-sycl-linker
+  ClangSYCLLinker.cpp
+
+  DEPENDS
+  ${tablegen_deps}
+  )
+
+set(CLANG_SYCL_LINKER_LIB_DEPS
+  clangBasic
+  )
+
+target_link_libraries(clang-sycl-linker
+  PRIVATE
+  ${CLANG_SYCL_LINKER_LIB_DEPS}
+  )
--- a/clang/tools/clang-sycl-linker/ClangSYCLLinker.cpp
+++ b/clang/tools/clang-sycl-linker/ClangSYCLLinker.cpp
@@ -0,0 +1,506 @@
+//=-------- clang-sycl-linker/ClangSYCLLinker.cpp - SYCL Linker util -------=//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===---------------------------------------------------------------------===//
+//
+// This tool executes a sequence of steps required to link device code in SYCL
+// device images. SYCL device code linking requires a complex sequence of steps
+// that include linking of llvm bitcode files, linking device library files
+// with the fully linked source bitcode file(s), running several SYCL specific
+// post-link steps on the fully linked bitcode file(s), and finally generating
+// target-specific device code.
+//===---------------------------------------------------------------------===//
+
+#include "clang/Basic/Version.h"
+
+#include "llvm/ADT/StringExtras.h"
+#include "llvm/BinaryFormat/Magic.h"
+#include "llvm/Bitcode/BitcodeWriter.h"
+#include "llvm/CodeGen/CommandFlags.h"
+#include "llvm/IR/DiagnosticPrinter.h"
+#include "llvm/IRReader/IRReader.h"
+#include "llvm/LTO/LTO.h"
+#include "llvm/Object/Archive.h"
+#include "llvm/Object/ArchiveWriter.h"
+#include "llvm/Object/Binary.h"
+#include "llvm/Object/ELFObjectFile.h"
+#include "llvm/Object/IRObjectFile.h"
+#include "llvm/Object/ObjectFile.h"
+#include "llvm/Object/OffloadBinary.h"
+#include "llvm/Option/ArgList.h"
+#include "llvm/Option/OptTable.h"
+#include "llvm/Option/Option.h"
+#include "llvm/Remarks/HotnessThresholdParser.h"
+#include "llvm/Support/CommandLine.h"
+#include "llvm/Support/FileOutputBuffer.h"
+#include "llvm/Support/FileSystem.h"
+#include "llvm/Support/InitLLVM.h"
+#include "llvm/Support/MemoryBuffer.h"
+#include "llvm/Support/Path.h"
+#include "llvm/Support/Program.h"
+#include "llvm/Support/Signals.h"
+#include "llvm/Support/StringSaver.h"
+#include "llvm/Support/TargetSelect.h"
+#include "llvm/Support/TimeProfiler.h"
+#include "llvm/Support/WithColor.h"
+
+using namespace llvm;
+using namespace llvm::opt;
+using namespace llvm::object;
+
+/// Save intermediary results.
+static bool SaveTemps = false;
+
+/// Print arguments without executing.
+static bool DryRun = false;
+
+/// Print verbose output.
+static bool Verbose = false;
+
+/// Filename of the output being created.
+static StringRef OutputFile;
+
+/// Directory to dump SPIR-V IR if requested by user.
+static SmallString<128> SPIRVDumpDir;
+
+static void printVersion(raw_ostream &OS) {
+  OS << clang::getClangToolFullVersion("clang-sycl-linker") << '\n';
+}
+
+/// The value of `argv[0]` when run.
+static const char *Executable;
+
+/// Temporary files to be cleaned up.
+static SmallVector<SmallString<128>> TempFiles;
+
+namespace {
+// Must not overlap with llvm::opt::DriverFlag.
+enum LinkerFlags { LinkerOnlyOption = (1 << 4) };
+
+enum ID {
+  OPT_INVALID = 0, // This is not an option ID.
+#define OPTION(...) LLVM_MAKE_OPT_ID(__VA_ARGS__),
+#include "SYCLLinkOpts.inc"
+  LastOption
+#undef OPTION
+};
+
+#define PREFIX(NAME, VALUE)                                                    \
+  static constexpr StringLiteral NAME##_init[] = VALUE;                        \
+  static constexpr ArrayRef<StringLiteral> NAME(NAME##_init,                   \
+                                                std::size(NAME##_init) - 1);
+#include "SYCLLinkOpts.inc"
+#undef PREFIX
+
+static constexpr OptTable::Info InfoTable[] = {
+#define OPTION(...) LLVM_CONSTRUCT_OPT_INFO(__VA_ARGS__),
+#include "SYCLLinkOpts.inc"
+#undef OPTION
+};
+
+class LinkerOptTable : public opt::GenericOptTable {
+public:
+  LinkerOptTable() : opt::GenericOptTable(InfoTable) {}
+};
+
+const OptTable &getOptTable() {
+  static const LinkerOptTable *Table = []() {
+    auto Result = std::make_unique<LinkerOptTable>();
+    return Result.release();
+  }();
+  return *Table;
+}
+
+[[noreturn]] void reportError(Error E) {
+  outs().flush();
+  logAllUnhandledErrors(std::move(E), WithColor::error(errs(), Executable));
+  exit(EXIT_FAILURE);
+}
+
+std::string getMainExecutable(const char *Name) {
+  void *Ptr = (void *)(intptr_t)&getMainExecutable;
+  auto COWPath = sys::fs::getMainExecutable(Name, Ptr);
+  return sys::path::parent_path(COWPath).str();
+}
+
+Expected<StringRef> createTempFile(const ArgList &Args, const Twine &Prefix,
+                                   StringRef Extension) {
+  SmallString<128> OutputFile;
+  if (Args.hasArg(OPT_save_temps)) {
+    // Generate a unique path name without creating a file
+    sys::fs::createUniquePath(Prefix + "-%%%%%%." + Extension, OutputFile,
+                              /*MakeAbsolute=*/false);
+  } else {
+    if (std::error_code EC =
+            sys::fs::createTemporaryFile(Prefix, Extension, OutputFile))
+      return createFileError(OutputFile, EC);
+  }
+
+  TempFiles.emplace_back(std::move(OutputFile));
+  return TempFiles.back();
+}
+
+Expected<std::string> findProgram(const ArgList &Args, StringRef Name,
+                                  ArrayRef<StringRef> Paths) {
+  if (Args.hasArg(OPT_dry_run))
+    return Name.str();
+  ErrorOr<std::string> Path = sys::findProgramByName(Name, Paths);
+  if (!Path)
+    Path = sys::findProgramByName(Name);
+  if (!Path)
+    return createStringError(Path.getError(),
+                             "Unable to find '" + Name + "' in path");
+  return *Path;
+}
+
+void printCommands(ArrayRef<StringRef> CmdArgs) {
+  if (CmdArgs.empty())
+    return;
+
+  llvm::errs() << " \"" << CmdArgs.front() << "\" ";
+  llvm::errs() << llvm::join(std::next(CmdArgs.begin()), CmdArgs.end(), " ")
+               << "\n";
+}
+
+/// Execute the command \p ExecutablePath with the arguments \p Args.
+Error executeCommands(StringRef ExecutablePath, ArrayRef<StringRef> Args) {
+  if (Verbose || DryRun)
+    printCommands(Args);
+
+  if (!DryRun)
+    if (sys::ExecuteAndWait(ExecutablePath, Args))
+      return createStringError(
+          "'%s' failed", sys::path::filename(ExecutablePath).str().c_str());
+  return Error::success();
+}
+
+Expected<SmallVector<std::string>> getInput(const ArgList &Args) {
+  // Collect all input bitcode files to be passed to llvm-link.
+  SmallVector<std::string> BitcodeFiles;
+  for (const opt::Arg *Arg : Args.filtered(OPT_INPUT)) {
+    std::optional<std::string> Filename = std::string(Arg->getValue());
+    if (!Filename || !sys::fs::exists(*Filename) ||
+        sys::fs::is_directory(*Filename))
+      continue;
+    file_magic Magic;
+    if (auto EC = identify_magic(*Filename, Magic))
+      return createStringError("Failed to open file " + *Filename);
+    // TODO: Current use case involves LLVM IR bitcode files as input.
+    // This will be extended to support objects and SPIR-V IR files.
+    if (Magic != file_magic::bitcode)
+      return createStringError("Unsupported file type");
+    BitcodeFiles.push_back(*Filename);
+  }
+  return BitcodeFiles;
+}
+
+/// Link all SYCL device input files into one before adding device library
+/// files. Device linking is performed using llvm-link tool.
+/// 'InputFiles' is the list of all LLVM IR device input files.
+/// 'Args' encompasses all arguments required for linking device code and will
+/// be parsed to generate options required to be passed into llvm-link.
+Expected<StringRef> linkDeviceInputFiles(ArrayRef<std::string> InputFiles,
+                                         const ArgList &Args) {
+  llvm::TimeTraceScope TimeScope("SYCL LinkDeviceInputFiles");
+
+  assert(InputFiles.size() && "No inputs to llvm-link");
+  // Early check to see if there is only one input.
+  if (InputFiles.size() < 2)
+    return InputFiles[0];
+
+  Expected<std::string> LLVMLinkPath =
+      findProgram(Args, "llvm-link", {getMainExecutable("llvm-link")});
+  if (!LLVMLinkPath)
+    return LLVMLinkPath.takeError();
+
+  SmallVector<StringRef> CmdArgs;
+  CmdArgs.push_back(*LLVMLinkPath);
+  for (auto &File : InputFiles)
+    CmdArgs.push_back(File);
+  // Create a new file to write the linked device file to.
+  auto OutFileOrErr =
+      createTempFile(Args, sys::path::filename(OutputFile), "bc");
+  if (!OutFileOrErr)
+    return OutFileOrErr.takeError();
+  CmdArgs.push_back("-o");
+  CmdArgs.push_back(*OutFileOrErr);
+  CmdArgs.push_back("--suppress-warnings");
+  if (Error Err = executeCommands(*LLVMLinkPath, CmdArgs))
+    return std::move(Err);
+  return *OutFileOrErr;
+}
+
+// This utility function is used to gather all SYCL device library files that
+// will be linked with input device files.
+// The list of files and its location are passed from driver.
+Expected<SmallVector<std::string>> getSYCLDeviceLibs(const ArgList &Args) {
+  SmallVector<std::string> DeviceLibFiles;
+  StringRef LibraryPath;
+  if (Arg *A = Args.getLastArg(OPT_library_path_EQ))
+    LibraryPath = A->getValue();
+  if (LibraryPath.empty())
+    return DeviceLibFiles;
+  if (Arg *A = Args.getLastArg(OPT_device_libs_EQ)) {
+    if (A->getValues().size() == 0)
+      return createStringError(
+          inconvertibleErrorCode(),
+          "Number of device library files cannot be zero.");
+    for (StringRef Val : A->getValues()) {
+      SmallString<128> LibName(LibraryPath);
+      llvm::sys::path::append(LibName, Val);
+      if (llvm::sys::fs::exists(LibName))
+        DeviceLibFiles.push_back(std::string(LibName));
+      else
+        return createStringError(inconvertibleErrorCode(),
+                                 "\'" + std::string(LibName) + "\'" +
+                                     " SYCL device library file is not found.");
+    }
+  }
+  return DeviceLibFiles;
+}
+
+/// Link all device library files and input file into one LLVM IR file. This
+/// linking is performed using llvm-link tool.
+/// 'InputFiles' is the list of all LLVM IR device input files.
+/// 'Args' encompasses all arguments required for linking device code and will
+/// be parsed to generate options required to be passed into llvm-link tool.
+static Expected<StringRef> linkDeviceLibFiles(StringRef InputFile,
+                                              const ArgList &Args) {
+  llvm::TimeTraceScope TimeScope("LinkDeviceLibraryFiles");
+
+  auto SYCLDeviceLibFiles = getSYCLDeviceLibs(Args);
+  if (!SYCLDeviceLibFiles)
+    return SYCLDeviceLibFiles.takeError();
+  if ((*SYCLDeviceLibFiles).empty())
+    return InputFile;
+
+  Expected<std::string> LLVMLinkPath =
+      findProgram(Args, "llvm-link", {getMainExecutable("llvm-link")});
+  if (!LLVMLinkPath)
+    return LLVMLinkPath.takeError();
+
+  // Create a new file to write the linked device file to.
+  auto OutFileOrErr =
+      createTempFile(Args, sys::path::filename(OutputFile), "bc");
+  if (!OutFileOrErr)
+    return OutFileOrErr.takeError();
+
+  SmallVector<StringRef, 8> CmdArgs;
+  CmdArgs.push_back(*LLVMLinkPath);
+  CmdArgs.push_back("-only-needed");
+  CmdArgs.push_back(InputFile);
+  for (auto &File : *SYCLDeviceLibFiles)
+    CmdArgs.push_back(File);
+  CmdArgs.push_back("-o");
+  CmdArgs.push_back(*OutFileOrErr);
+  CmdArgs.push_back("--suppress-warnings");
+  if (Error Err = executeCommands(*LLVMLinkPath, CmdArgs))
+    return std::move(Err);
+  return *OutFileOrErr;
+}
+
+/// Add any llvm-spirv option that relies on a specific Triple in addition
+/// to user supplied options.
+static void getSPIRVTransOpts(const ArgList &Args,
+                              SmallVector<StringRef, 8> &TranslatorArgs,
+                              const llvm::Triple Triple) {
+  // Enable NonSemanticShaderDebugInfo.200 for non-Windows
+  const bool IsWindowsMSVC =
+      Triple.isWindowsMSVCEnvironment() || Args.hasArg(OPT_is_windows_msvc_env);
+  const bool EnableNonSemanticDebug = !IsWindowsMSVC;
+  if (EnableNonSemanticDebug) {
+    TranslatorArgs.push_back(
+        "-spirv-debug-info-version=nonsemantic-shader-200");
+  } else {
+    TranslatorArgs.push_back("-spirv-debug-info-version=ocl-100");
+    // Prevent crash in the translator if input IR contains DIExpression
+    // operations which don't have mapping to OpenCL.DebugInfo.100 spec.
+    TranslatorArgs.push_back("-spirv-allow-extra-diexpressions");
+  }
+  std::string UnknownIntrinsics("-spirv-allow-unknown-intrinsics=llvm.genx.");
+
+  TranslatorArgs.push_back(Args.MakeArgString(UnknownIntrinsics));
+
+  // Disable all the extensions by default
+  std::string ExtArg("-spirv-ext=-all");
+  std::string DefaultExtArg =
+      ",+SPV_EXT_shader_atomic_float_add,+SPV_EXT_shader_atomic_float_min_max"
+      ",+SPV_KHR_no_integer_wrap_decoration,+SPV_KHR_float_controls"
+      ",+SPV_KHR_expect_assume,+SPV_KHR_linkonce_odr";
+  std::string INTELExtArg =
+      ",+SPV_INTEL_subgroups,+SPV_INTEL_media_block_io"
+      ",+SPV_INTEL_device_side_avc_motion_estimation"
+      ",+SPV_INTEL_fpga_loop_controls,+SPV_INTEL_unstructured_loop_controls"
+      ",+SPV_INTEL_fpga_reg,+SPV_INTEL_blocking_pipes"
+      ",+SPV_INTEL_function_pointers,+SPV_INTEL_kernel_attributes"
+      ",+SPV_INTEL_io_pipes,+SPV_INTEL_inline_assembly"
+      ",+SPV_INTEL_arbitrary_precision_integers"
+      ",+SPV_INTEL_float_controls2,+SPV_INTEL_vector_compute"
+      ",+SPV_INTEL_fast_composite"
+      ",+SPV_INTEL_arbitrary_precision_fixed_point"
+      ",+SPV_INTEL_arbitrary_precision_floating_point"
+      ",+SPV_INTEL_variable_length_array,+SPV_INTEL_fp_fast_math_mode"
+      ",+SPV_INTEL_long_constant_composite"
+      ",+SPV_INTEL_arithmetic_fence"
+      ",+SPV_INTEL_global_variable_decorations"
+      ",+SPV_INTEL_cache_controls"
+      ",+SPV_INTEL_fpga_buffer_location"
+      ",+SPV_INTEL_fpga_argument_interfaces"
+      ",+SPV_INTEL_fpga_invocation_pipelining_attributes"
+      ",+SPV_INTEL_fpga_latency_control"
+      ",+SPV_INTEL_task_sequence"
+      ",+SPV_KHR_shader_clock"
+      ",+SPV_INTEL_bindless_images";
+  ExtArg = ExtArg + DefaultExtArg + INTELExtArg;
+  ExtArg += ",+SPV_INTEL_token_type"
+            ",+SPV_INTEL_bfloat16_conversion"
+            ",+SPV_INTEL_joint_matrix"
+            ",+SPV_INTEL_hw_thread_queries"
+            ",+SPV_KHR_uniform_group_instructions"
+            ",+SPV_INTEL_masked_gather_scatter"
+            ",+SPV_INTEL_tensor_float32_conversion"
+            ",+SPV_INTEL_optnone"
+            ",+SPV_KHR_non_semantic_info"
+            ",+SPV_KHR_cooperative_matrix";
+  TranslatorArgs.push_back(Args.MakeArgString(ExtArg));
+}
+
+/// Run LLVM to SPIR-V translation.
+/// Converts 'File' from LLVM bitcode to SPIR-V format using llvm-spirv tool.
+/// 'Args' encompasses all arguments required for linking device code and will
+/// be parsed to generate options required to be passed into llvm-spirv tool.
+static Expected<StringRef> runLLVMToSPIRVTranslation(StringRef File,
+                                                     const ArgList &Args) {
+  llvm::TimeTraceScope TimeScope("LLVMToSPIRVTranslation");
+  StringRef LLVMSPIRVPath = Args.getLastArgValue(OPT_llvm_spirv_path_EQ);
+  Expected<std::string> LLVMToSPIRVProg =
+      findProgram(Args, "llvm-spirv", {LLVMSPIRVPath});
+  if (!LLVMToSPIRVProg)
+    return LLVMToSPIRVProg.takeError();
+
+  SmallVector<StringRef, 8> CmdArgs;
+  CmdArgs.push_back(*LLVMToSPIRVProg);
+  const llvm::Triple Triple(Args.getLastArgValue(OPT_triple));
+  getSPIRVTransOpts(Args, CmdArgs, Triple);
+  StringRef LLVMToSPIRVOptions;
+  if (Arg *A = Args.getLastArg(OPT_llvm_spirv_options_EQ))
+    LLVMToSPIRVOptions = A->getValue();
+  LLVMToSPIRVOptions.split(CmdArgs, " ", /* MaxSplit = */ -1,
+                           /* KeepEmpty = */ false);
+  CmdArgs.append({"-o", OutputFile});
+  CmdArgs.push_back(File);
+  if (Error Err = executeCommands(*LLVMToSPIRVProg, CmdArgs))
+    return std::move(Err);
+
+  if (!SPIRVDumpDir.empty()) {
+    std::error_code EC =
+        llvm::sys::fs::create_directory(SPIRVDumpDir, /*IgnoreExisting*/ true);
+    if (EC)
+      return createStringError(
+          EC,
+          formatv("failed to create dump directory. path: {0}, error_code: {1}",
+                  SPIRVDumpDir, EC.value()));
+
+    StringRef Path = OutputFile;
+    StringRef Filename = llvm::sys::path::filename(Path);
+    SmallString<128> CopyPath = SPIRVDumpDir;
+    CopyPath.append(Filename);
+    EC = llvm::sys::fs::copy_file(Path, CopyPath);
+    if (EC)
+      return createStringError(
+          EC,
+          formatv(
+              "failed to copy file. original: {0}, copy: {1}, error_code: {2}",
+              Path, CopyPath, EC.value()));
+  }
+
+  return OutputFile;
+}
+
+Error runSYCLLink(ArrayRef<std::string> Files, const ArgList &Args) {
+  llvm::TimeTraceScope TimeScope("SYCLDeviceLink");
+  // First llvm-link step
+  auto LinkedFile = linkDeviceInputFiles(Files, Args);
+  if (!LinkedFile)
+    reportError(LinkedFile.takeError());
+
+  // second llvm-link step
+  auto DeviceLinkedFile = linkDeviceLibFiles(*LinkedFile, Args);
+  if (!DeviceLinkedFile)
+    reportError(DeviceLinkedFile.takeError());
+
+  // LLVM to SPIR-V translation step
+  auto SPVFile = runLLVMToSPIRVTranslation(*DeviceLinkedFile, Args);
+  if (!SPVFile)
+    return SPVFile.takeError();
+  return Error::success();
+}
+
+} // namespace
+
+int main(int argc, char **argv) {
+  InitLLVM X(argc, argv);
+
+  Executable = argv[0];
+  sys::PrintStackTraceOnErrorSignal(argv[0]);
+
+  const OptTable &Tbl = getOptTable();
+  BumpPtrAllocator Alloc;
+  StringSaver Saver(Alloc);
+  auto Args = Tbl.parseArgs(argc, argv, OPT_INVALID, Saver, [&](StringRef Err) {
+    reportError(createStringError(inconvertibleErrorCode(), Err));
+  });
+
+  if (Args.hasArg(OPT_help) || Args.hasArg(OPT_help_hidden)) {
+    Tbl.printHelp(
+        outs(), "clang-sycl-linker [options] <options to sycl link steps>",
+        "A utility that wraps around several steps required to link SYCL "
+        "device files.\n"
+        "This enables LLVM IR linking, post-linking and code generation for "
+        "SYCL targets.",
+        Args.hasArg(OPT_help_hidden), Args.hasArg(OPT_help_hidden));
+    return EXIT_SUCCESS;
+  }
+
+  if (Args.hasArg(OPT_version))
+    printVersion(outs());
+
+  Verbose = Args.hasArg(OPT_verbose);
+  DryRun = Args.hasArg(OPT_dry_run);
+  SaveTemps = Args.hasArg(OPT_save_temps);
+
+  OutputFile = "a.spv";
+  if (Args.hasArg(OPT_o))
+    OutputFile = Args.getLastArgValue(OPT_o);
+
+  if (Args.hasArg(OPT_spirv_dump_device_code_EQ)) {
+    Arg *A = Args.getLastArg(OPT_spirv_dump_device_code_EQ);
+    SmallString<128> Dir(A->getValue());
+    if (Dir.empty())
+      llvm::sys::path::native(Dir = "./");
+    else
+      Dir.append(llvm::sys::path::get_separator());
+
+    SPIRVDumpDir = Dir;
+  }
+
+  // Get the input files to pass to the linking stage.
+  auto FilesOrErr = getInput(Args);
+  if (!FilesOrErr)
+    reportError(FilesOrErr.takeError());
+
+  // Run SYCL linking process on the generated inputs.
+  if (Error Err = runSYCLLink(*FilesOrErr, Args))
+    reportError(std::move(Err));
+
+  // Remove the temporary files created.
+  if (!Args.hasArg(OPT_save_temps))
+    for (const auto &TempFile : TempFiles)
+      if (std::error_code EC = sys::fs::remove(TempFile))
+        reportError(createFileError(TempFile, EC));
+
+  return EXIT_SUCCESS;
+}
--- a/clang/tools/clang-sycl-linker/SYCLLinkOpts.td
+++ b/clang/tools/clang-sycl-linker/SYCLLinkOpts.td
@@ -0,0 +1,52 @@
+include "llvm/Option/OptParser.td"
+
+def LinkerOnlyOption : OptionFlag;
+
+def help : Flag<["-", "--"], "help">,
+  HelpText<"Display available options (--help-hidden for more)">;
+
+def help_hidden : Flag<["-", "--"], "help-hidden">,
+  HelpText<"Display all available options">;
+
+def verbose : Flag<["-"], "v">, HelpText<"Print verbose information">;
+def version : Flag<["--"], "version">,
+  HelpText<"Display the version number and exit">;
+
+def o : JoinedOrSeparate<["-"], "o">, MetaVarName<"<path>">,
+  HelpText<"Path to file to write output">;
+def output : Separate<["--"], "output-file">, Alias<o>, Flags<[HelpHidden]>,
+  HelpText<"Alias for -o">;
+
+def library_path_EQ : Joined<["--", "-"], "library-path=">,
+  Flags<[HelpHidden]>, HelpText<"Add <dir> to the library search path">;
+
+def device_libs_EQ : CommaJoined<["--", "-"], "device-libs=">,
+  Flags<[LinkerOnlyOption]>,
+  HelpText<"A comma separated list of device libraries that are linked during the device link.">;
+
+def triple : Joined<["--"], "triple">,
+  HelpText<"The device target triple">;
+def arch : Separate<["--", "-"], "arch">,
+  HelpText<"Specify the name of the target architecture.">;
+
+def save_temps : Flag<["--", "-"], "save-temps">,
+  Flags<[LinkerOnlyOption]>, HelpText<"Save intermediate results">;
+
+def dry_run : Flag<["--", "-"], "dry-run">, Flags<[LinkerOnlyOption]>,
+  HelpText<"Print generated commands without running.">;
+
+def spirv_dump_device_code_EQ : Joined<["--", "-"], "spirv-dump-device-code=">,
+  Flags<[LinkerOnlyOption]>,
+  HelpText<"Path to the folder where the tool dumps SPIR-V device code. Other formats aren't dumped.">;
+
+def is_windows_msvc_env : Flag<["--", "-"], "is-windows-msvc-env">,
+  Flags<[LinkerOnlyOption, HelpHidden]>;
+
+def llvm_spirv_path_EQ : Joined<["--"], "llvm-spirv-path=">,
+  Flags<[LinkerOnlyOption]>, MetaVarName<"<dir>">,
+  HelpText<"Set the system llvm-spirv path">;
+
+// Options to pass to llvm-spirv tool
+def llvm_spirv_options_EQ : Joined<["--", "-"], "llvm-spirv-options=">,
+  Flags<[LinkerOnlyOption]>,
+  HelpText<"Options that will control llvm-spirv step">;