llvm][docs] Use default checkout location in test suite guide (#139264)
Step 2 tells you to checkout "llvm-test-suite" to "test-suite", but I don't see a particular reason to use a non-default path. If you're following the instructions exactly, it all works, but if you autopilot that step it is surprising later when things do not work. It's not hard for an individual to fix later, but we should suggest the least surprising thing where we can.
This commit is contained in:
@@ -41,7 +41,7 @@ Quickstart
|
|||||||
2. Check out the `test-suite` module with:
|
2. Check out the `test-suite` module with:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
% git clone https://github.com/llvm/llvm-test-suite.git test-suite
|
% git clone https://github.com/llvm/llvm-test-suite.git
|
||||||
```
|
```
|
||||||
|
|
||||||
3. Create a build directory and use CMake to configure the suite. Use the
|
3. Create a build directory and use CMake to configure the suite. Use the
|
||||||
@@ -53,8 +53,8 @@ Quickstart
|
|||||||
% mkdir test-suite-build
|
% mkdir test-suite-build
|
||||||
% cd test-suite-build
|
% cd test-suite-build
|
||||||
% cmake -DCMAKE_C_COMPILER=<path to llvm build>/bin/clang \
|
% cmake -DCMAKE_C_COMPILER=<path to llvm build>/bin/clang \
|
||||||
-C../test-suite/cmake/caches/O3.cmake \
|
-C../llvm-test-suite/cmake/caches/O3.cmake \
|
||||||
../test-suite
|
../llvm-test-suite
|
||||||
```
|
```
|
||||||
|
|
||||||
**NOTE!** if you are using your built clang, and you want to build and run the
|
**NOTE!** if you are using your built clang, and you want to build and run the
|
||||||
@@ -102,9 +102,9 @@ MicroBenchmarks/XRay microbenchmarks, you need to add `compiler-rt` to your
|
|||||||
# Make sure pandas and scipy are installed. Prepend `sudo` if necessary.
|
# Make sure pandas and scipy are installed. Prepend `sudo` if necessary.
|
||||||
% pip install pandas scipy
|
% pip install pandas scipy
|
||||||
# Show a single result file:
|
# Show a single result file:
|
||||||
% test-suite/utils/compare.py results.json
|
% llvm-test-suite/utils/compare.py results.json
|
||||||
# Compare two result files:
|
# Compare two result files:
|
||||||
% test-suite/utils/compare.py results_a.json results_b.json
|
% llvm-test-suite/utils/compare.py results_a.json results_b.json
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
||||||
@@ -287,7 +287,7 @@ benchmarks. CMake can print a list of them:
|
|||||||
|
|
||||||
Generate build files for the ninja build tool.
|
Generate build files for the ninja build tool.
|
||||||
|
|
||||||
- `-Ctest-suite/cmake/caches/<cachefile.cmake>`
|
- `-Cllvm-test-suite/cmake/caches/<cachefile.cmake>`
|
||||||
|
|
||||||
Use a CMake cache. The test-suite comes with several CMake caches which
|
Use a CMake cache. The test-suite comes with several CMake caches which
|
||||||
predefine common or tricky build configurations.
|
predefine common or tricky build configurations.
|
||||||
@@ -304,7 +304,7 @@ Example usage:
|
|||||||
- Basic Usage:
|
- Basic Usage:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
% test-suite/utils/compare.py baseline.json
|
% llvm-test-suite/utils/compare.py baseline.json
|
||||||
Warning: 'test-suite :: External/SPEC/CINT2006/403.gcc/403.gcc.test' has No metrics!
|
Warning: 'test-suite :: External/SPEC/CINT2006/403.gcc/403.gcc.test' has No metrics!
|
||||||
Tests: 508
|
Tests: 508
|
||||||
Metric: exec_time
|
Metric: exec_time
|
||||||
@@ -328,14 +328,14 @@ Example usage:
|
|||||||
- Show compile_time or text segment size metrics:
|
- Show compile_time or text segment size metrics:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
% test-suite/utils/compare.py -m compile_time baseline.json
|
% llvm-test-suite/utils/compare.py -m compile_time baseline.json
|
||||||
% test-suite/utils/compare.py -m size.__text baseline.json
|
% llvm-test-suite/utils/compare.py -m size.__text baseline.json
|
||||||
```
|
```
|
||||||
|
|
||||||
- Compare two result files and filter short running tests:
|
- Compare two result files and filter short running tests:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
% test-suite/utils/compare.py --filter-short baseline.json experiment.json
|
% llvm-test-suite/utils/compare.py --filter-short baseline.json experiment.json
|
||||||
...
|
...
|
||||||
Program baseline experiment diff
|
Program baseline experiment diff
|
||||||
|
|
||||||
@@ -349,7 +349,7 @@ Example usage:
|
|||||||
runtime each:
|
runtime each:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
% test-suite/utils/compare.py base0.json base1.json base2.json vs exp0.json exp1.json exp2.json
|
% llvm-test-suite/utils/compare.py base0.json base1.json base2.json vs exp0.json exp1.json exp2.json
|
||||||
```
|
```
|
||||||
|
|
||||||
### Continuous Tracking with LNT
|
### Continuous Tracking with LNT
|
||||||
@@ -365,11 +365,11 @@ External Suites
|
|||||||
|
|
||||||
External suites such as SPEC can be enabled by either
|
External suites such as SPEC can be enabled by either
|
||||||
|
|
||||||
- placing (or linking) them into the `test-suite/test-suite-externals/xxx` directory (example: `test-suite/test-suite-externals/speccpu2000`)
|
- placing (or linking) them into the `llvm-test-suite/test-suite-externals/xxx` directory (example: `llvm-test-suite/test-suite-externals/speccpu2000`)
|
||||||
- using a configuration option such as `-D TEST_SUITE_SPEC2000_ROOT=path/to/speccpu2000`
|
- using a configuration option such as `-D TEST_SUITE_SPEC2000_ROOT=path/to/speccpu2000`
|
||||||
|
|
||||||
You can find further information in the respective README files such as
|
You can find further information in the respective README files such as
|
||||||
`test-suite/External/SPEC/README`.
|
`llvm-test-suite/External/SPEC/README`.
|
||||||
|
|
||||||
For the SPEC benchmarks you can switch between the `test`, `train` and
|
For the SPEC benchmarks you can switch between the `test`, `train` and
|
||||||
`ref` input datasets via the `TEST_SUITE_RUN_TYPE` configuration option.
|
`ref` input datasets via the `TEST_SUITE_RUN_TYPE` configuration option.
|
||||||
@@ -398,7 +398,7 @@ picked up automatically if placed into a subdirectory of the test-suite or when
|
|||||||
setting the `TEST_SUITE_SUBDIRS` variable:
|
setting the `TEST_SUITE_SUBDIRS` variable:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
% cmake -DTEST_SUITE_SUBDIRS=path/to/my/benchmark-suite ../test-suite
|
% cmake -DTEST_SUITE_SUBDIRS=path/to/my/benchmark-suite ../llvm-test-suite
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
||||||
@@ -416,7 +416,7 @@ Example:
|
|||||||
% cmake -DTEST_SUITE_PROFILE_GENERATE=ON \
|
% cmake -DTEST_SUITE_PROFILE_GENERATE=ON \
|
||||||
-DTEST_SUITE_USE_IR_PGO=ON \
|
-DTEST_SUITE_USE_IR_PGO=ON \
|
||||||
-DTEST_SUITE_RUN_TYPE=train \
|
-DTEST_SUITE_RUN_TYPE=train \
|
||||||
../test-suite
|
../llvm-test-suite
|
||||||
% make
|
% make
|
||||||
% llvm-lit .
|
% llvm-lit .
|
||||||
# Use the profile data for compilation and actual benchmark run:
|
# Use the profile data for compilation and actual benchmark run:
|
||||||
@@ -446,7 +446,7 @@ information can be found here:
|
|||||||
- [https://cmake.org/cmake/help/latest/manual/cmake-toolchains.7.html](https://cmake.org/cmake/help/latest/manual/cmake-toolchains.7.html)
|
- [https://cmake.org/cmake/help/latest/manual/cmake-toolchains.7.html](https://cmake.org/cmake/help/latest/manual/cmake-toolchains.7.html)
|
||||||
|
|
||||||
Cross compilation from macOS to iOS is possible with the
|
Cross compilation from macOS to iOS is possible with the
|
||||||
`test-suite/cmake/caches/target-target-*-iphoneos-internal.cmake` CMake cache
|
`llvm-test-suite/cmake/caches/target-target-*-iphoneos-internal.cmake` CMake cache
|
||||||
files; this requires an internal iOS SDK.
|
files; this requires an internal iOS SDK.
|
||||||
|
|
||||||
### Running
|
### Running
|
||||||
@@ -464,10 +464,10 @@ There are two ways to run the tests in a cross compilation setting:
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
% cmake -G Ninja -D CMAKE_C_COMPILER=path/to/clang \
|
% cmake -G Ninja -D CMAKE_C_COMPILER=path/to/clang \
|
||||||
-C ../test-suite/cmake/caches/target-arm64-iphoneos-internal.cmake \
|
-C ../llvm-test-suite/cmake/caches/target-arm64-iphoneos-internal.cmake \
|
||||||
-D CMAKE_BUILD_TYPE=Release \
|
-D CMAKE_BUILD_TYPE=Release \
|
||||||
-D TEST_SUITE_REMOTE_HOST=mydevice \
|
-D TEST_SUITE_REMOTE_HOST=mydevice \
|
||||||
../test-suite
|
../llvm-test-suite
|
||||||
% ninja
|
% ninja
|
||||||
% ninja rsync
|
% ninja rsync
|
||||||
% llvm-lit -j1 -o result.json .
|
% llvm-lit -j1 -o result.json .
|
||||||
|
|||||||
Reference in New Issue
Block a user