Reading and Understanding the Swift Driver Source Code

November 25, 2017

In the previous article of this running series on the Swift compiler, I introduced the Swift compiler driver. To recap: the driver is the executable that is run when users invoke swift or swiftc on the command line. It's responsible for splitting up a single swift or swiftc invocation into multiple child jobs. These jobs are command line invocations of swift -frontend, ld, or other command-line utilities.

This article explains the details of how the Swift driver executable and libswiftDriver accomplish this. Specifically, I'll explain the code that is executed when I run swiftc hello.swift on the command line.

I've found that understanding this process has made it easier for me to fix Swift JIRA bugs labeled "Driver". Please let me know if this article helps you do the same!

The six stages of the Swift driver

I think of the driver as having six main stages, in which it:

1. Checks whether it needs to be split up: The driver checks whether the current invocation of swift or swiftc should be split up into jobs, or whether it's currently being run as one of those jobs. In other words, it checks whether it's being run as swift or swiftc (and needs to be split up), or as swift -frontend (and so has already been split up). In our case, swiftc hello.swift is not an invocation of swift -frontend, and so the driver will determine it needs to be split up.
2. Instantiates a swift::Driver: If the invocation needs to be split up into jobs by the driver, a swift::Driver object is instantiated, and its intializer determines the "driver mode" (explained in the previous article).
3. Instantiates a swift::Compilation: Assuming a driver mode of swiftc, the swift::Driver::buildCompilation method is called. This parses command-line options to create a swift::Compilation object, which encapsulates the information required to build out a set of jobs. One of the most important pieces of information stored at this point is a mapping of inputs to desired outputs.
4. Splits up the work of a Compilation into a graph of swift::Action objects: The swift::Driver::buildActions method creates a graph of swift::Action objects. These represent smaller unit of work, such as "compile this Swift source file into an object" and "link these objects into an executable." The actions for any given swift invocation can be examined with the -driver-print-actions option.
5. Instantiates a list of swift::Job based on those actions: Actions contain all the information necessary for a swift::Compilation to instantiate a swift::Job, which represents the actual command-line invocation used to run a child job. To translate actions into jobs, the swift::Compilation::buildJobs method is invoked. As explained in the previous article, the jobs for any given swift invocation can be exampled with the -driver-print-jobs option.
6. Executes each of the list of swift::Job: Finally, the swift::Compilation::petformJobs method is called. This method runs each of the jobs within a task queue.

The apple/swift repository includes a document named Driver Design & Internals that describes the Swift 2.0 driver as being composed of four stages. I prefer to think of it as six stages, because I like to break the stages down more granularly – my stages one through three correspond to that document's stage one.

Stage 1: Checking whether the current swift invocation needs to be split up by the driver

I'll begin where the swift executable (and every C-based program) begins: the main function.

It's important to note that I'll elide many lines of source code that aren't immediately relevant to the six stages of the driver. For example, the very first thing the driver does is call the INITIALIZE_LLVM macro. The Swift compiler would not function without this line of code, but because it's not crucial to understanding the Swift driver, I won't go into that macro here.

1.1: Checking if the invocation is trying to run a "subcommand"

One of the first things swift does in main is check whether it's being run as a "subcommand":

111  int main(int argc_, const char **argv_) {
...
124    StringRef ExecName = llvm::sys::path::stem(argv[0]);
125    SmallString<256> SubcommandName;
126    if (shouldRunAsSubcommand(ExecName, SubcommandName, argv)) {

"Subcommands" here to refer to commands like swift build or swift test. The shouldRunAsSubcommand function checks the first two command-line arguments and, if they fit a certain pattern, it returns an executable name based on them. For example, an invocation that begins with swift build would result in this function returning "swift-build":

 70  static bool shouldRunAsSubcommand(StringRef ExecName,
 71                                    SmallString<256> &SubcommandName,
 72                                    SmallVectorImpl<const char *> &Args) {
 ..
 78    if (ExecName != "swift")
 79      return false;
 ..  
 87    StringRef FirstArg(Args[1]);
 88    if (FirstArg.startswith("-") || FirstArg.find('.') != StringRef::npos ||
 89        FirstArg.find('/') != StringRef::npos)
 90      return false;
...  
103    SubcommandName.assign("swift-");
104    SubcommandName.append(Subcommand);
105  
106    return true;
107  }

There are two things to note here:

1. The Swift driver only considers invocations that begin with swift as potential subcommand invocations. So swiftc build would not be considered a subcommand.
2. Invocations whose second argument contains a period, or that start with a dash, as not considered subcommands. So swift hello.swift</span> would not be evaluated as a subcommand.

If the driver determines that it is running a subcommand, such as swift build, it finds an executable named swift-build in the same directory as the swift or swiftc executable currently being run, and it executes it and immediately exits the program by calling the swift::ExecuteInPlace function:

111  int main(int argc_, const char **argv_) {
...
126    if (shouldRunAsSubcommand(ExecName, SubcommandName, argv)) {
...
129      SmallString<256> SubcommandPath(
130        llvm::sys::path::parent_path(getExecutablePath(argv[0])));
131      llvm::sys::path::append(SubcommandPath, SubcommandName);
...
146      argv.push_back(nullptr);
147      ExecuteInPlace(SubcommandPath.c_str(), argv.data());
...
154    }

This is a fun quirk about how the Swift compiler works: if you invoke it with swift foo or swift hello, it'll interpret those as subcommand invocations. This means the driver will attempt to invoke an executable named swift-foo or swift-hello. You can try this yourself by creating an executable named swift-goodjob next to your built Swift executable:

/path/to/your/swift-macosx-x86_64/bin/swift-goodjob

#!/usr/bin/env bash
echo "Keep up the good work!"

Make the file executable by running chmod a+x /path/to/your/swift-macosx-x86_64/bin/swift-goodjob on the command line, and you'll be able to invoke it as a Swift subcommand:

swift goodjob

Invoking it results in a reaffirming message:

Keep up the good work!

1.2: Checking if the invocation is for "integrated tools" like swift -frontend

When I invoke swiftc hello.swift on the command line, it's not evaluated as a subcommand for two reasons:

1. I'm invoking swiftc, not swift. Only swift <subcommand-name> invocations are considered potential subcommands.
2. The second argument, hello.swift, contains a period. The code I listed above explicitly discounts invocations whose second command contains a period.

Since swiftc hello.swift isn't considered a subcommand, the driver would continue by checking whether swiftc hello.swift needs to be split up into child jobs. The only case in which this isn't necessary is when swift or swiftc is being invoked as swift -frontend, swift -modulewrap, or swift -apinotes. Each of these are considered to have already been split up.

Therefore one of the first things the Swift driver does, after checking whether it should execute a subcommand, is check whether it should execute the swift -frontend code path:

111  int main(int argc_, const char **argv_) {
...    // ...checks whether to run a subcommand.
...
158    StringRef FirstArg(argv[1]);
159    if (FirstArg == "-frontend") {
160      return performFrontend(llvm::makeArrayRef(argv.data()+2,
161                                                argv.data()+argv.size()),
162                             argv[0], (void *)(intptr_t)getExecutablePath);
163    }
164    if (FirstArg == "-modulewrap") {
165      return modulewrap_main(llvm::makeArrayRef(argv.data()+2,
166                                                argv.data()+argv.size()),
167                             argv[0], (void *)(intptr_t)getExecutablePath);
168    }
169    if (FirstArg == "-apinotes") {
170      return apinotes_main(llvm::makeArrayRef(argv.data()+1,
171                                              argv.data()+argv.size()));
172    }

The first code path in the if statement, for swift -frontend or swiftc -frontend, calls the performFrontend function. This kicks off everything I think of when I think of "compiling Swift code": parsing the source code, building an abstract syntax tree, type checking, and so on.

I'll cover the frontend in a future article. For now, the driver has determined that my swiftc hello.swift invocation probably needs to be split up, and so it continues to stage two.

Stage 2: Instantiating a swift::Driver

The logic behind splitting up my swiftc hello.swift invocation is handled by an instance of swift::Driver, which the code in main proceeds to instantiate:

111  int main(int argc_, const char **argv_) {
...    // ...checks whether to run a subcommand,
...    // or whether to execute the compiler frontend.
182
183    Driver TheDriver(Path, ExecName, argv, Diags);

The Driver initializer does two things:

1. It instantiates a table of Swift option flags. (In order to do the explanation justice, I'll describe option tables and option parsing in a future article.)
2. It determines the "driver mode" it's being run under. Recall that the previous article explained how the name of the executable (and the --driver-mode= option) is used to determine how the driver behaves.

64  Driver::Driver(StringRef DriverExecutable,
65                 StringRef Name,
66                 ArrayRef<const char *> Args,
67                 DiagnosticEngine &Diags)
68    : Opts(createSwiftOptTable()), Diags(Diags),
69      Name(Name), DriverExecutable(DriverExecutable),
70      DefaultTargetTriple(llvm::sys::getDefaultTargetTriple()) {
..
74    parseDriverKind(Args.slice(1));
75  }

Once the Driver is instantiated, the driver executable determines what to do based on its driver mode: a mode of swift-format results in an early exit after calling swift_format_main, for example.

111  int main(int argc_, const char **argv_) {
...    // ...checks whether to run a subcommand,
...    // or whether to execute the compiler frontend.
182
183    Driver TheDriver(Path, ExecName, argv, Diags);
184    switch (TheDriver.getDriverKind()) {
...
189    case Driver::DriverKind::SwiftFormat:
190      return swift_format_main(
191        TheDriver.getArgsWithoutProgramNameAndDriverMode(argv),
192        argv[0], (void *)(intptr_t)getExecutablePath);
193    default:
194      break;
195    }

If you're interested in how swift-format works, just follow the code in the swift_format_main function to find out. swiftc hello.swift, on the other hand, results in a driver mode of DriverKind::Batch, and so this early return isn't taken. Instead, the driver continues its work of splitting up my invocation. To do so, it'll create a swift::Compilation object.

Stage 3: Instantiating a swift::Compilation

The driver uses an instance of swift::Compilation to build up and a store a list of jobs. To begin, it calls the swift::Driver::buildCompilation method:

111  int main(int argc_, const char **argv_) {
...    // ...checks whether to run a subcommand,
...    // or whether to execute the compiler frontend,
...    // and then instantiates a Driver object and
...    // checks whether to return early for swift-format.
196  
197    std::unique_ptr<Compilation> C = TheDriver.buildCompilation(argv);

3.1: Parsing and validating command-line arguments

The Driver::buildCompilation method parses the command-line arguments passed into the original swift or swiftc invocation, by calling the Driver::parseArgStrings method. It then validates those arguments. For example, the -incremental and -whole-module-optimization options are mutually exclusive, so it prints a warning if they're used together:

504  std::unique_ptr<Compilation> Driver::buildCompilation(
505      ArrayRef<const char *> Args) {
...
510    std::unique_ptr<InputArgList> ArgList(parseArgStrings(Args.slice(1)));
...
529    bool Incremental = ArgList->hasArg(options::OPT_incremental);
530    if (ArgList->hasArg(options::OPT_whole_module_optimization)) {
531      if (Incremental && ShowIncrementalBuildDecisions) {
532        llvm::outs() << "Incremental compilation has been disabled, because it "
533                     << "is not compatible with whole module optimization.";
534      }
535      Incremental = false;
536    }

3.2: Instantiating a swift::ToolChain

Once the arguments have been validated, the Driver::buildCompilation method instantiates a swift::ToolChain object. A swift::ToolChain is an object that's capable of translating "actions" (swift::Action) into "jobs" (swift::Job).

swift::ToolChain itself is an abstract base class. Several subclasses exist to create jobs for specific targets. For example, the swift::toolchains::Darwin toolchain knows how to create jobs that would work when compiling for a macOS or iOS target. swift::toolchains::Android knows how to create jobs for Android targets.

An appropriate toolchain is instantiated from within the Driver::buildCompilation method, by calling the makeToolChain function:

504  std::unique_ptr<Compilation> Driver::buildCompilation(
505      ArrayRef<const char *> Args) {
...    // ...parses and validates arguments.
562
563    std::unique_ptr<const ToolChain> TC =
564        makeToolChain(*this, llvm::Triple(DefaultTargetTriple));

The makeToolChain function determines which toolchain to instantiate based on the target triple:

204  static std::unique_ptr<const ToolChain>
205  makeToolChain(Driver &driver, const llvm::Triple &target) {
206    switch (target.getOS()) {
207    case llvm::Triple::Darwin:
208    case llvm::Triple::MacOSX:
209    case llvm::Triple::IOS:
210    case llvm::Triple::TvOS:
211    case llvm::Triple::WatchOS:
212      return llvm::make_unique<toolchains::Darwin>(driver, target);
213      break;
214    case llvm::Triple::Linux:
215      if (target.isAndroid()) {
216        return llvm::make_unique<toolchains::Android>(driver, target);
217      } else {
218        return llvm::make_unique<toolchains::GenericUnix>(driver, target);
219      }
220      break;
...
232    }
233  }

3.3: Determining the swift::driver::OutputInfo

The driver's freshly instantiated ToolChain is capable of translating actions into jobs. But to determine which actions my swiftc hello.swift invocation is composed of, the driver needs to construct an OutputInfo object:

504  std::unique_ptr<Compilation> Driver::buildCompilation(
505      ArrayRef<const char *> Args) {
...    // ...parses arguments and instantiates a toolchain.
...
583    OutputInfo OI;
584    buildOutputInfo(*TC, *TranslatedArgList, Inputs, OI);

The Driver::buildOutputInfo method sets properties on the OutputInfo object, based on the driver mode and the presence of driver arguments like -emit-executable or -emit-library. Among the most important of these is OutputInfo::LinkAction and OutputInfo::CompilerOutputType:

1052  void Driver::buildOutputInfo(const ToolChain &TC, const DerivedArgList &Args,
1053                               const InputFileList &Inputs,
1054                               OutputInfo &OI) const {
....
1090      switch (OutputModeArg->getOption().getID()) {
1091      case options::OPT_emit_executable:
1092        OI.LinkAction = LinkKind::Executable;
1093        OI.CompilerOutputType = types::TY_Object;
1094        break;
1095  
1096      case options::OPT_emit_library:
1097        OI.LinkAction = LinkKind::DynamicLibrary;
1098        OI.CompilerOutputType = types::TY_Object;
1099        break;
....
1334  }

Finally, the driver is ready to build a list of actions, by instantiating a Compilation object, and then calling the Driver::buildActions method:

504  std::unique_ptr<Compilation> Driver::buildCompilation(
505      ArrayRef<const char *> Args) {
...    // ...parses arguments, instantiates a toolchain,
...    // and builds output info.
688
689    std::unique_ptr<Compilation> C(new Compilation(Diags, Level,
690                                                   std::move(ArgList),
691                                                   std::move(TranslatedArgList),
692                                                   std::move(Inputs),
693                                                   ArgsHash, StartTime,
694                                                   NumberOfParallelCommands,
695                                                   Incremental,
696                                                   DriverSkipExecution,
697                                                   SaveTemps,
698                                                   ShowDriverTimeCompilation,
699                                                   std::move(StatsReporter)));
700    // Construct the graph of Actions.
701    SmallVector<const Action *, 8> TopLevelActions;
702    buildActions(TopLevelActions, *TC, OI, OFM.get(),
703                 rebuildEverything ? nullptr : &outOfDateMap, *C);

Stage 4: Instantiating a graph of swift::Action

ToolChain objects are capable of translating "actions" – high-level descriptions of the inputs and outputs of a Swift compilation – into concrete "jobs".

The previous article explained that invoking swiftc hello.swift -driver-print-jobs prints these "jobs", which are themselves command-line invocations:

/Users/bgesiak/Source/apple/build/Ninja-ReleaseAssert/swift-macosx-x86_64/bin/swift \
    -frontend -c -primary-file hello.swift \
    -target x86_64-apple-darwin16.7.0 \
    -enable-objc-interop -color-diagnostics -module-name hello \
    -o /var/folders/ry/2ryfdsb56b30092626qprw6d3rb3ss/T/hello-6ab6fd.o

/usr/bin/ld \
    /var/folders/ry/2ryfdsb56b30092626qprw6d3rb3ss/T/hello-6ab6fd.o \
    -lobjc -lSystem -arch x86_64 -L /Users/bgesiak/Source/apple/build/Ninja-ReleaseAssert/swift-macosx-x86_64/lib/swift/macosx \
    -rpath /Users/bgesiak/Source/apple/build/Ninja-ReleaseAssert/swift-macosx-x86_64/lib/swift/macosx \
    -macosx_version_min 10.12.0 -no_objc_category_merging \
    -o hello

Besides jobs, the driver is also capable of printing a list of actions. Invoking swiftc hello.swift -driver-print-actions outputs:

0: input, "hello.swift", swift
1: compile, {0}, object
2: link, {1}, image

This looks similar to the output of swiftc hello.swift -driver-print-jobs, with a few key differences:

1. There are three actions. These are "translated," by a ToolChain, into two jobs.
2. The input file hello.swift is represented as an action, but there is no equivalent representation of it as a job.
3. The actions are high-level descriptions of what needs to happen. For example, 1: compile, {0}, object means "input file hello.swift needs to be compiled into an object." The job that corresponds to that action is a swift -frontend -c invocation with tons of arguments. In other words, the job is the concrete way that the toolchain chose to fulfill the action.

The actions themselves are constructed from within the Driver::buildActions method. This is a long method that's responsible for creating a list of actions based on the OutputInfo instance, the Driver object's DriverKind, and in some cases the target platform. I'll skip over most of it to show just the parts that are relevant to my swiftc hello.swift invocation:

1336  void Driver::buildActions(SmallVectorImpl<const Action *> &TopLevelActions,
1337                            const ToolChain &TC, const OutputInfo &OI,
1338                            const OutputFileMap *OFM,
1339                            const InputInfoMap *OutOfDateMap,
1340                            Compilation &C) const {
....
1349    SmallVector<const Action *, 2> AllModuleInputs;
1350    SmallVector<const Action *, 2> AllLinkerInputs;
1351
1352    switch (OI.CompilerMode) {
1353    case OutputInfo::Mode::StandardCompile: {
....  
1378      for (const InputPair &Input : Inputs) {
1379        types::ID InputType = Input.first;
1380        const Arg *InputArg = Input.second;
1381  
1382        Action *Current = C.createAction<InputAction>(*InputArg, InputType);
1383        switch (InputType) {
1384        case types::TY_Swift:
1385        case types::TY_SIL:
1386        case types::TY_SIB: {
....
1405            Current = C.createAction<CompileJobAction>(Current,
1406                                                       OI.CompilerOutputType,
1407                                                       previousBuildState);
....
1410            AllModuleInputs.push_back(Current);
1411          }
1412          AllLinkerInputs.push_back(Current);
1413          break;
1414        }
....  
1572    if (OI.shouldLink() && !AllLinkerInputs.empty()) {
1573      auto *LinkAction = C.createAction<LinkJobAction>(AllLinkerInputs,
1574                                                       OI.LinkAction);
....
1627    }
1628  }

The gist of the Driver::buildActions method is that it:

1. Loops over each of the inputs (in my case, that's just hello.swift) and creates an InputAction for each of them. The Compilation object stores a list of its actions internally.
2. For source code inputs of type .swift, .sil, or .sib, it creates a CompileJobAction, also stored internally to the Compilation object. It also specifies that the result of a CompileJobAction should be treated as an input to a linker action.
3. If a link action has been requested, and if there are inputs to that link action, the driver creates a LinkJobAction. Not all swiftc invocations result in a linker invocation – for example, had I ran swiftc hello.swift -emit-object, then the OutputInfo::shouldLink method would return false, and no LinkJobAction would have been created.

Stage 5: Instantiating a list of swift::Job based on those actions

After constructing the actions, the driver calls the Driver::buildJobs method to translate actions into jobs:

504  std::unique_ptr<Compilation> Driver::buildCompilation(
505      ArrayRef<const char *> Args) {
...    // ...parses arguments, instantiates a toolchain,
...    // and builds output info, instantiates a Compilation,
...    // and builds a graph of actions.
...
713    buildJobs(TopLevelActions, OI, OFM.get(), *TC, *C);
...
752    return C;
753  }

The Driver::buildJobs method loops over each Action and passes any JobAction objects to the Driver::buildJobsForAction method. This filters out inputs, which are represented as InputAction and so are not instances of JobAction:

1673  void Driver::buildJobs(ArrayRef<const Action *> TopLevelActions,
1674                         const OutputInfo &OI, const OutputFileMap *OFM,
1675                         const ToolChain &TC, Compilation &C) const {
....
1702    for (const Action *A : TopLevelActions) {
1703      if (auto *JA = dyn_cast<JobAction>(A)) {
1704        (void)buildJobsForAction(C, JA, OI, OFM, TC, /*TopLevel*/true, JobCache);
1705      }
1706    }
1707  }

The Driver::buildJobsForAction method does a lot. Among other things, it records dependencies between actions, in order to improve Swift's incremental compilation. That's a topic I'll need to write several articles about, so I won't attempt to cover it here.

At its core, though, the Driver::buildJobsForAction method calls through to the ToolChain object that was instantiated in stage three above, in order to translate each Action into a Job. It then adds that Job to a list stored within the Compilation object:

1993  Job *Driver::buildJobsForAction(Compilation &C, const JobAction *JA,
1994                                  const OutputInfo &OI,
1995                                  const OutputFileMap *OFM,
1996                                  const ToolChain &TC, bool AtTopLevel,
1997                                  JobCacheMap &JobCache) const {
....
2291    std::unique_ptr<Job> ownedJob = TC.constructJob(*JA, C, std::move(InputJobs),
2292                                                    InputActions, 
2293                                                    std::move(Output), OI);
2294    Job *J = C.addJob(std::move(ownedJob));
....  
2371    return J;
2372  }

In order to translate an action into a job, the ToolChain base class uses a macro that calls the appropriate ToolChain::constructInvocation method for each action class type. ToolChain classes overload constructInvocation in order to execute separate logic for, say, constructInvocation(const CompileJobAction &, ...) and constructInvocation(const LinkJobAction &, ...):

 66  std::unique_ptr<Job>
 67  ToolChain::constructJob(const JobAction &JA,
 68                          Compilation &C,
 69                          SmallVectorImpl<const Job *> &&inputs,
 70                          ArrayRef<const Action *> inputActions,
 71                          std::unique_ptr<CommandOutput> output,
 72                          const OutputInfo &OI) const {
 ..
 75    auto invocationInfo = [&]() -> InvocationInfo {
 76      switch (JA.getKind()) {
 77    #define CASE(K) case Action::K: \
 78        return constructInvocation(cast<K##Action>(JA), context);
 79      CASE(CompileJob)
 80      CASE(InterpretJob)
 81      CASE(BackendJob)
 82      CASE(MergeModuleJob)
 83      CASE(ModuleWrapJob)
 84      CASE(LinkJob)
 ..
 97    }();
...  
120    return llvm::make_unique<Job>(JA, std::move(inputs), std::move(output),
121                                  executablePath,
122                                  std::move(invocationInfo.Arguments),
123                                  std::move(invocationInfo.ExtraEnvironment),
124                                  std::move(invocationInfo.FilelistInfo));
125  }

Each subclass of toolchain – swift::toolchains::Darwin, swift::toolchains::Android, etc. – constructs platform-specific command-line invocations for each action. Some actions, however, are the same on each platform. One example of such an action is a CompileJobAction – jobs for these are invocations of swift -frontend, and these are pretty much the same on any platform. As a result, they're handled in the base ToolChain class.

Here's how the base ToolChain class transforms the compile action for hello.swift into an invocation of swift -frontend -c:

200  ToolChain::InvocationInfo
201  ToolChain::constructInvocation(const CompileJobAction &job,
202                                 const JobContext &context) const {
203    InvocationInfo II{SWIFT_EXECUTABLE_NAME};
204    ArgStringList &Arguments = II.Arguments;
205  
206    Arguments.push_back("-frontend");
...
210    switch (context.OI.CompilerMode) {
211    case OutputInfo::Mode::StandardCompile:
212    case OutputInfo::Mode::SingleCompile: {
213      switch (context.Output.getPrimaryOutputType()) {
214      case types::TY_Object:
215        FrontendModeOption = "-c";
216        break;
...
280    }
...
500    return II;
501  }

Unlike compile actions, link actions differ greatly depending on the target platform. As a result, most ToolChain subclasses override the ToolChain::constructInvocation(const LinkJobAction &, ...) method. Here's swift::toolchains::Darwin, which adds object files as inputs, conditionally adds the -dylib option when linking a dynamic library, and specifies the output file with -o:

1211  ToolChain::InvocationInfo
1212  toolchains::Darwin::constructInvocation(const LinkJobAction &job,
1213                                          const JobContext &context) const {
....
1228    const char *LD = "ld";
....  
1238    InvocationInfo II = {LD};
1239    ArgStringList &Arguments = II.Arguments;
....  
1250    addInputsOfType(Arguments, context.InputActions, types::TY_Object);
....  
1264    switch (job.getKind()) {
....
1270    case LinkKind::DynamicLibrary:
1271      Arguments.push_back("-dylib");
1272      break;
1273    }
....
1463    Arguments.push_back("-o");
1464    Arguments.push_back(context.Output.getPrimaryOutputFilename().c_str());
1465  
1466    return II;
1467  }

Stage 6: Executing each of the list of swift::Job

Recall that this article began in the swift executable's main function, the entry point of the compiler. That function eventually called the Driver::buildCompilation method, in stage three. The Driver::buildCompilation method called Driver::buildActions in stage four, and then Driver::buildJobs in stage five, before returning back to main.

Now back in main, the driver has a Compilation object that has a complete list of the Job objects it needs to execute. Finally, the driver calls the Compilation::performJobs method, which kicks off the execution of those jobs:

111  int main(int argc_, const char **argv_) {
...    // ...checks whether to run a subcommand,
...    // or whether to execute the compiler frontend,
...    // and then instantiates a Driver object and
...    // checks whether to return early for swift-format,
...    // and finally builds a Compilation object.
...
203    return C->performJobs();
...  
207  }

The Compilation::performJobs method just calls through to the Compilation::performJobsImpl method, which in turn schedules the jobs and executes them on a task queue:

826  int Compilation::performJobsImpl(bool &abnormalExit) {
827    PerformJobsState State(*this);
828  
829    State.scheduleInitialJobs();
830    State.scheduleAdditionalJobs();
831    State.runTaskQueueToCompletion();
832    State.checkUnfinishedJobs();
...
843    return State.getResult();
844  }

The task queue itself is implemented in one of two ways, depending on whether the driver is running on a Darwin or on a Unix host:

26  #if LLVM_ON_UNIX && !defined(__CYGWIN__) && !defined(__HAIKU__)
27  #include "Unix/TaskQueue.inc"
28  #else
29  #include "Default/TaskQueue.inc"
30  #endif

I won't go into the details of each of these implementations in this article. Suffice it to say that the child job invocations are each executed in order.

Putting it all together: the story of how swiftc hello.swift is executed

In summary, here's how an invocation of swiftc hello.swift, when run on my macOS host machine, goes through the six stages of the Swift driver:

1. The driver determines the invocation isn't a subcommand (those start with "swift"), and it isn't a swift -frontend invocation, so it must need to be split up somehow.
2. The driver instantiates a swift::Driver object in order to handle splitting up swiftc hello.swift. Because the name of the executable is swiftc, the driver infers that it's being run in DriverKind::Batch mode.
3. The driver instantiates a swift::Compilation object to store each of the actions and jobs that will be constructed. A swift::toolchains::Darwin toolchain object is instantiated, so that it can be used later to translate actions into macOS-specific jobs. The swiftc hello.swift invocation arguments are parsed in order to construct a swift::driver::OutputInfo object whose LinkAction is LinkKind::Executable, and whose CompilerOutputType is types::TY_Object.
4. The driver constructs a graph of swift::Action objects based on the OutputInfo. In the case of swiftc hello.swift, an InputAction is created to represent hello.swift, a CompileJobAction is created based on an OutputInfo::CompilerOutputType of TY_Object, and a LinkJobAction is created because the OutputInfo::LinkAction was set to Executable.
5. The driver invokes the toolchain in order to instantiate a swift::Job for each JobAction – so that's the CompileJobAction and LinkJobAction from the previous stage.
6. The driver executes these two jobs on a task queue, thereby compiling hello.swift into an object named hello.o, and linking hello.o into an executable named hello.

I hope that sheds some light on how the Swift driver behaves. Despite the length of this article, there's lot I didn't cover. For example, the Swift driver is also responsible for incremental compilation – if it determines that the result of a job is already available from a previous compilation, it won't run that job. I'll try to write more about how this works in a future article.