How the driver finds anything

Source references in this chapter are to GCC 15.2.0 (releases/gcc-15.2.0).

Your cross compiler cannot find crt1.o. You add --sysroot=/arm-rootfs, and now it can. So you try the same trick on crtbegin.o, which is also missing, and nothing happens — it is still missing, and --sysroot has no effect on it whatever.

Both files arrive on the link line through the same directive, %s. Both are looked up in the same list. And yet one follows your sysroot and the other ignores it.

The answer is that the list is built from entries registered by two different functions, and only one of them prepends the sysroot. This chapter is about how that list gets built, in what order, and which flag touches which part of it.

Two lists, not one

The driver keeps two search lists (three, counting headers, which get a chapter of their own):

$ gcc -print-search-dirs

They are genuinely separate lists in separate variables, and a flag can feed one, the other, or both. -L feeds only the second. -B feeds all three, which is why it is the blunt instrument.

Everything the linker eventually searches comes out of startfile_prefixes, via one directive:

 %D	Dump out a -L option for each directory in startfile_prefixes.
	If multilib_dir is set, extra entries are generated with it affixed.

Listing 5-1: %D (gcc/gcc.cc:592-593; handler at :6250).

So the driver's "library search path" is not a path it searches. It is a list it converts into -L flags. The driver never opens a library file. It emits -l and -L and lets ld do the resolving. The one exception is -print-file-name, which walks the list itself precisely so that it can tell you what ld would conclude.

The order the list is built in

startfile_prefixes is assembled during startup, and the order of the calls is the search order. In sequence:

1. -B prefixes, from the command line (gcc.cc:4590).
2. GCC_EXEC_PREFIX, from the environment (gcc.cc:4784).
3. LIBRARY_PATH, from the environment — native compilers only (gcc.cc:4923).
4. The relocated install prefix, computed from argv[0] (gcc.cc:4818-4835).
5. The tool directory, $prefix/<target>/lib/ (gcc.cc:5537-5539).
6. Either STARTFILE_PREFIX_SPEC (gcc.cc:8579-8587) or the STANDARD_STARTFILE_PREFIX* chain (gcc.cc:8590-8630) — never both.

That ordering is documented, in the internals manual rather than the user manual: gcc/doc/tm.texi:570 begins "Here is the order of prefixes tried for startfiles" and enumerates them, including the conditions. If you only remember one manual reference from Part I, info gccint 'Target Macros' Driver is a good candidate.

Let us take the interesting steps in turn.

-B wins, and nothing can sort before it

    case OPT_B:
      {
	size_t len = strlen (arg);

	/* Catch the case where the user has forgotten to append a
	   directory separator to the path.  Note, they may be using
	   -B to add an executable name prefix, eg "i386-elf-", in
	   order to distinguish between multiple installations of
	   GCC in the same directory.  Hence we must check to see
	   if appending a directory separator actually makes a
	   valid directory name.  */
	if (!IS_DIR_SEPARATOR (arg[len - 1])
	    && is_directory (arg))
	  {
	    ...
	  }

	add_prefix (&exec_prefixes, arg, NULL,
		    PREFIX_PRIORITY_B_OPT, 0, 0);
	add_prefix (&startfile_prefixes, arg, NULL,
		    PREFIX_PRIORITY_B_OPT, 0, 0);
	add_prefix (&include_prefixes, arg, NULL,
		    PREFIX_PRIORITY_B_OPT, 0, 0);
      }

Listing 5-2: -B (gcc/gcc.cc:4590-4619).

Three things in one listing.

-B feeds all three lists. That is the whole of its definition. It is not a library path option or an executable path option; it is a prefix applied to programs, link inputs and headers simultaneously, which is exactly why it fixes problems you did not know you had and breaks things you did not expect.

The trailing / is added for you — but only conditionally, and the comment explains why. A -B argument without a trailing separator is a legitimate executable name prefix, so the driver has to check whether appending a separator would name a real directory before doing it. If you write -B/opt/alt and /opt/alt exists, you get /opt/alt/. If it does not exist, you have just told the driver that your programs are called /opt/altcc1 and so on.

PREFIX_PRIORITY_B_OPT is the highest priority there is. And "highest" is stronger than it sounds, because the enum has exactly two members:

/* Ranking of prefixes in the sort list. -B prefixes are put before
   all others.  */

enum path_prefix_priority
{
  PREFIX_PRIORITY_B_OPT,
  PREFIX_PRIORITY_LAST
};

Listing 5-3: the priority enum (gcc/gcc.cc:3120-3127).

Two values. So -B beats the environment, and the environment beats the compiled-in defaults, and there is no way to express "before -B". Every environment contribution and every compiled-in default lands at PREFIX_PRIORITY_LAST and is ordered only by when it was added.

One thing -B does not do: it does not move C++ header search. It adds GCC's own header directories, which is enough to change which <stdint.h> you compile against but not which <vector>. That asymmetry is Chapter 1.9's and Chapter 1.12's territory. And it has nothing to do with the linker's -Bstatic.

Relocation: how a moved toolchain still works

You can move an installed GCC tree to a completely different directory and it keeps working. That is not luck, and it is not RPATH. The driver recomputes its own prefix from argv[0] on every single run:

  gcc_libexec_prefix = standard_libexec_prefix;
#ifndef VMS
  /* FIXME: make_relative_prefix doesn't yet work for VMS.  */
  if (!gcc_exec_prefix)
    {
      gcc_exec_prefix = get_relative_prefix (decoded_options[0].arg,
					     standard_bindir_prefix,
					     standard_exec_prefix);
      gcc_libexec_prefix = get_relative_prefix (decoded_options[0].arg,
					     standard_bindir_prefix,
					     standard_libexec_prefix);
      if (gcc_exec_prefix)
	xputenv (concat ("GCC_EXEC_PREFIX=", gcc_exec_prefix, NULL));
    }

Listing 5-4: relocation (gcc/gcc.cc:4822-4835). get_relative_prefix is make_relative_prefix from libiberty, assigned at :4816.

Read the three arguments. It knows where it was configured to be installed (standard_bindir_prefix, standard_exec_prefix), it knows where it actually is (decoded_options[0].arg, which is argv[0]), and it computes the same relative relationship against the new location. If you configured with --prefix=/opt/gcc-arm and the binary is now at /home/me/toolchain/bin/gcc, then standard_exec_prefix is recomputed as /home/me/toolchain/lib/gcc/.

Notice the xputenv on the last line. The driver exports the answer as GCC_EXEC_PREFIX, whether or not you set that variable yourself. So relocation and the environment variable are two faces of one mechanism, and the driver's own comment a few lines later says so:

  /* From this point onward, gcc_exec_prefix is non-null if the toolchain
     is relocated. The toolchain was either relocated using GCC_EXEC_PREFIX
     or an automatically created GCC_EXEC_PREFIX from
     decoded_options[0].arg.  */

Listing 5-5: the driver's summary of its own state (gcc/gcc.cc:4855-4858).

That variable, and the fact that setting it merely pre-empts this computation, is Chapter 1.6's subject.

There is a caveat the manual states and that people get wrong. Relocation moves GCC's own tree. It does not in general move your sysroot — that only happens when the sysroot was configured to live inside the exec prefix, in which case configure defines an extra macro and the driver relocates it too (gcc/configure.ac:176-182, acted on at gcc/gcc.cc:5542-5559). The manual's wording:

If the specified directory is a subdirectory of ${exec_prefix}, then it will be found relative to the GCC binaries if the installation tree is moved.

— gcc/doc/install.texi:2720-2722

Bare --with-sysroot with no argument defaults to $exec_prefix/<target>/sys-root (gcc/configure.ac:169-170), which satisfies that condition — so the default is relocatable and an explicit --with-sysroot=/arm-rootfs is not. That is a good default and an easy surprise.

The tool directory

Two lines put the target's own subtree on the lists:

  add_prefix (&exec_prefixes,
	      concat (tooldir_prefix, "bin", dir_separator_str, NULL),
	      "BINUTILS", PREFIX_PRIORITY_LAST, 0, 0);
  add_prefix (&startfile_prefixes,
	      concat (tooldir_prefix, "lib", dir_separator_str, NULL),
	      "BINUTILS", PREFIX_PRIORITY_LAST, 0, 1);

Listing 5-6: the tool directory (gcc/gcc.cc:5534-5539).

$prefix/<target>/bin/ for programs, $prefix/<target>/lib/ for libraries. Note add_prefix, not add_sysrooted_prefix — the tool directory is never sysrooted. This is the single most important line for bare-metal toolchains: it is why, after you install newlib into $prefix/<target>/lib/, -lc simply works with no -L and no sysroot. And it is why, in a cross install, libstdc++.so is found there rather than in your sysroot no matter what --sysroot you pass.

The final argument, 1, marks this prefix as an OS multilib prefix, which changes how it gets expanded. That is Chapter 1.7.

The startfile chain, and the cross-compiler gate

The last step is the one with a branch in it. Either the target defines STARTFILE_PREFIX_SPEC, in which case it wins outright:

  /* Look for startfiles in the standard places.  */
  if (*startfile_prefix_spec != 0
      && do_spec_2 (startfile_prefix_spec, NULL) == 0
      && do_spec_1 (" ", 0, NULL) == 0)
    {
      for (const char *arg : argbuf)
	add_sysrooted_prefix (&startfile_prefixes, arg, "BINUTILS",
			      PREFIX_PRIORITY_LAST, 0, 1);
    }

Listing 5-7: the STARTFILE_PREFIX_SPEC branch (gcc/gcc.cc:8579-8587).

or the standard chain runs — and it is guarded:

  /* We should eventually get rid of all these and stick to
     startfile_prefix_spec exclusively.  */
  else if (*cross_compile == '0' || target_system_root)

Listing 5-8: the gate (gcc/gcc.cc:8588-8590).

That one line is worth staring at. It says: add the standard system library directories only if this is a native compiler, or a sysroot is configured.

A cross compiler with no sysroot gets no /lib and no /usr/lib at all.

This is deliberate and correct. Searching the host's /usr/lib for target libraries is how you get baffling file in wrong format errors from the linker, or worse, a link that succeeds against completely wrong objects. If your cross compiler seems to have suspiciously few entries in libraries:, this line is why, and it is protecting you.

Inside the gate, the entries that make it in:

      /* Sysrooted prefixes are relocated because target_system_root is
	 also relocated by gcc_exec_prefix.  */
      if (*standard_startfile_prefix_1)
 	add_sysrooted_prefix (&startfile_prefixes,
			      standard_startfile_prefix_1, "BINUTILS",
			      PREFIX_PRIORITY_LAST, 0, 1);
      if (*standard_startfile_prefix_2)
	add_sysrooted_prefix (&startfile_prefixes,
			      standard_startfile_prefix_2, "BINUTILS",
			      PREFIX_PRIORITY_LAST, 0, 1);

Listing 5-9: where the sysroot gets glued on (gcc/gcc.cc:8621-8630).

And those two prefixes are exactly what you would guess:

#ifndef STANDARD_STARTFILE_PREFIX_1
#define STANDARD_STARTFILE_PREFIX_1 "/lib/"
#endif
#ifndef STANDARD_STARTFILE_PREFIX_2
#define STANDARD_STARTFILE_PREFIX_2 "/usr/lib/"
#endif

Listing 5-10: the two standard prefixes (gcc/gcc.cc:1613-1618).

add_sysrooted_prefix, from Chapter 1.2, is where /lib/ becomes /arm-rootfs/lib/. That call is the entire sysroot mechanism for libraries. Nothing else applies it, nothing can retroactively apply it, and the entries registered with plain add_prefix — the tool directory, -B, GCC_EXEC_PREFIX, the relocated install prefix — are never sysrooted at all.

Which answers the question this chapter opened with. crt1.o lives in the sysroot's /usr/lib, reached through Listing 5-9. crtbegin.o lives in libsubdir, reached through the relocated install prefix registered with add_prefix. Same %s, same list, different registration function, different answer to --sysroot.

There is one more piece of the machine-dependent chain worth noting, because it explains why native compilers behave differently: MD_EXEC_PREFIX, MD_STARTFILE_PREFIX and MD_STARTFILE_PREFIX_1 are #undef'd outright for a cross compiler, with the comment "Don't use these prefixes for a cross compiler" (gcc/gcc.cc:1620-1624).

The linker gets told about the sysroot too

Prefixes are not the only route. If ld supports it, the driver passes the sysroot straight through, so that ld's own built-in search directories and any =-prefixed path inside a linker script become sysroot-relative:

#ifdef HAVE_LD_SYSROOT
  /* Pass the --sysroot option to the linker, if it supports that.  If
     there is a sysroot_suffix_spec, it has already been processed by
     this point, so target_system_root really is the system root we
     should be using.  */
  if (target_system_root)
    {
      obstack_grow (&obstack, "%(sysroot_spec) ", strlen ("%(sysroot_spec) "));
      obstack_grow0 (&obstack, link_spec, strlen (link_spec));
      set_spec ("link", XOBFINISH (&obstack, const char *), false);
    }
#endif

Listing 5-11: prepending the sysroot to the link spec (gcc/gcc.cc:8554-8566).

Note what that does: it rewrites the link spec at startup, prepending a reference to another spec. And that other spec is one line:

#ifndef SYSROOT_SPEC
# define SYSROOT_SPEC "--sysroot=%R"
#endif

Listing 5-12: SYSROOT_SPEC (gcc/gcc.cc:1189-1191).

%R expands to the sysroot, from Chapter 1.4. So --sysroot=/arm-rootfs on your command line becomes --sysroot=/arm-rootfs on ld's command line, and you can see it:

$ gcc --sysroot=/arm-rootfs -### hello.c 2>&1 | grep -o '\-\-sysroot=[^ "]*'

Listing 5-11 is also a nice demonstration of Chapter 1.4's claim that %G and friends name mutable slots. Here the driver is mutating the link slot at startup, by name, through set_spec.

Which knob do I want?

Four flags look like "where GCC looks for things" and are four different mechanisms. Two are configure-time and two are usage-time.

The two usage-time ones are what this chapter has been about. --sysroot overwrites target_system_root (gcc.cc:4559), which every add_sysrooted_prefix call and every %R then picks up.

The configure-time half belongs to Part II, but one thing is worth saying now because it explains a category of bug. --with-build-sysroot sets the sysroot used only while building the target libraries, and it is refreshingly literal:

AC_ARG_WITH(build-sysroot,
  [...],
  [if test x"$withval" != x ; then
     SYSROOT_CFLAGS_FOR_TARGET="--sysroot=$withval"
   fi], ...)

Listing 5-13: --with-build-sysroot (gcc/configure.ac:140-146).

It is an extra --sysroot= appended to the command line the build uses to compile libgcc and libstdc++. It leaves no trace in the installed compiler. The reason you need it is that libstdc++'s configure runs several hundred compile-and-link probes against the target C library, and if those cannot link, features configure themselves to not available — so you get a quietly crippled libstdc++ rather than a build failure. Part III returns to this.

Diagnosing it

The whole chapter reduces to four commands and one table.

$ arm-linux-gnueabihf-gcc -print-sysroot              # the baked-in sysroot
$ arm-linux-gnueabihf-gcc -print-search-dirs          # the two lists
$ arm-linux-gnueabihf-gcc -print-file-name=crt1.o     # the libc half
$ arm-linux-gnueabihf-gcc -print-file-name=crtbegin.o # the libgcc half

And do not forget that three environment variables feed these same lists invisibly. Nothing on your command line reveals them, so when a toolchain behaves differently in CI than on your desk:

$ gcc -print-search-dirs
$ env -u GCC_EXEC_PREFIX -u COMPILER_PATH -u LIBRARY_PATH gcc -print-search-dirs

That is the next chapter.

Documentation coverage

The startfile prefix order is genuinely well documented, in the internals manual at gcc/doc/tm.texi:570. -B and --sysroot are in Directory Options (gcc/doc/invoke.texi:19484), and the configure-time flags in install.texi.

Two things are not documented:

• That add_prefix versus add_sysrooted_prefix is what decides whether a directory follows --sysroot. The manual lists which prefixes get "any sysroot modifications" per entry, which is the same information, but it never says the mechanism is per-entry and irreversible.
• That -print-search-dirs shows the list before multilib expansion.

Next: the same lists, moved invisibly from the environment.

All source references in this chapter are to GCC 15.2.0 (releases/gcc-15.2.0). Line numbers in other releases will differ; the surrounding code rarely does. Where behaviour itself changed across a major version, it is flagged inline.