package_index generates kzips for Kythe to use in generating cross-references in Code Search.
Googlers: For more information on package_index and its usage, refer to the documentation on the ChOps Eng portal.
To build and run package_index,
$ go build $ ./package_index \ --checkout_dir ~/chromium/src \ --out_dir src/out/Debug \ --path_to_compdb ~/chromium/src/out/Debug/compile_commands.json \ --path_to_gn_targets ~/chromium/src/out/Debug/gn_targets.json \ --keep_filepaths_files --corpus chromium.googlesource.com/chromium/src \ --build_config linux \ --path_to_archive_output ~/test.kzip \
You can test package_index by running
$ go test -run TestPackageIndexUnix # or TestPackageIndexWindows
This script is invoked as part of the gen builder, right towards the end. It takes as input various artifacts produced during earlier parts of the build.
Let's walk through this line-by-line, to explain what each part is for, and
what we must do for the necessary input to exist. Note that here we assume you
have the build repo checked out under
This points to the root of your Chromium checkout. Again, we assume a particular location, modify as necessary. Any commands listed later on should be run from this directory, unless otherwise specified.
We need to build chromium before running this script, as we package up
generated files into our kzip. Any build config will work, but
used for the production pipeline, so we use it here too.
If you're interested in Mojom, you need to add
enable_kythe_annotations = true to your
args.gn, and then run
$ infra/build/scripts/slave/recipe_modules/codesearch/resources/add_kythe_metadata.py ~/chromium/src/out/Debug/gen
for the relevant annotations to be present. If not, you can skip this step.
We read the compilation database to determine which files are built and with what args. You can generate this file by running the following:
$ ninja -C out/Debug chromium -t compdb cc cxx objc objcxx > out/Debug/compile_commands.json
We also read a dump of the gn targets to find all the Mojom targets, as their build rules aren't as straightforward. This can be generated by running the following:
$ gn desc out/Debug '*' --format=json > out/Debug/gn_targets.json
We use "filepaths" files to find all the transitive includes for each file. By
default they're deleted each time, and this parameter disables that, which is
useful if you want to run the script multiple times without recreating these
files each time. To generate these files, you first must download the
translation_unit clang tool:
Then you run this tool over all the files of interest via:
$ tools/clang/scripts/run_tool.py --tool translation_unit -p out/Debug
Note that this reads from the compilation database, so you must generate this as described above before running this tool.
These are parameters written into fields of the kzip units. Their values are arbitrary strings, the values given here match the production configuration.
Finally, we specify where the output should be written.
Uploading to CIPD
If building for linux, run
GOOS=linux GOARCH=amd64 vpython ../build.py --upload package_index.yaml
Or, if building for windows, run
GOOS=windows GOARCH=amd64 vpython ../build.py --upload package_index.yaml
You may need to check CIPD to make sure the instances uploaded are set to the latest ref.
There is no documentation for this package.