Summary:
Recent changes added an include path in the float128 type that used the
internal `libc` path to find the macro. This doesn't work once it's
installed because we need to search from the root of the install dir.
This patch adds "include/" to the include path so that our inclusion
of installed headers always match the internal use.
This patch adds the r, R, k, and K conversion specifiers to printf, with
accompanying tests. They are guarded behind the
LIBC_COPT_PRINTF_DISABLE_FIXED_POINT flag as well as automatic fixed
point support detection.
Summary:
This is a massive patch because it reworks the entire build and
everything that depends on it. This is not split up because various bots
would fail otherwise. I will attempt to describe the necessary changes
here.
This patch completely reworks how the GPU build is built and targeted.
Previously, we used a standard runtimes build and handled both NVPTX and
AMDGPU in a single build via multi-targeting. This added a lot of
divergence in the build system and prevented us from doing various
things like building for the CPU / GPU at the same time, or exporting
the startup libraries or running tests without a full rebuild.
The new appraoch is to handle the GPU builds as strict cross-compiling
runtimes. The first step required
https://github.com/llvm/llvm-project/pull/81557 to allow the `LIBC`
target to build for the GPU without touching the other targets. This
means that the GPU uses all the same handling as the other builds in
`libc`.
The new expected way to build the GPU libc is with
`LLVM_LIBC_RUNTIME_TARGETS=amdgcn-amd-amdhsa;nvptx64-nvidia-cuda`.
The second step was reworking how we generated the embedded GPU library
by moving it into the library install step. Where we previously had one
`libcgpu.a` we now have `libcgpu-amdgpu.a` and `libcgpu-nvptx.a`. This
patch includes the necessary clang / OpenMP changes to make that not
break the bots when this lands.
We unfortunately still require that the NVPTX target has an `internal`
target for tests. This is because the NVPTX target needs to do LTO for
the provided version (The offloading toolchain can handle it) but cannot
use it for the native toolchain which is used for making tests.
This approach is vastly superior in every way, allowing us to treat the
GPU as a standard cross-compiling target. We can now install the GPU
utilities to do things like use the offload tests and other fun things.
Some certain utilities need to be built with
`--target=${LLVM_HOST_TRIPLE}` as well. I think this is a fine
workaround as we
will always assume that the GPU `libc` is a cross-build with a
functioning host.
Depends on https://github.com/llvm/llvm-project/pull/81557
The inf and nan string index bounds checks were after the index was
being used. This patch moves the index usage to the end of the
condition.
Fixes#79988
The semantics for casting can range from "bitcast" (same representation)
to "different representation", to "type promotion". Here we remove the
cast operator and force usage of `get_val` as the only function to get
the floating point value, making the intent clearer and more consistent.
Summary:
The `puts` function consists of an initial write and then another write
to append the newline. When executing code in parallel, it is possible
for these writes to becomes disjointed. This code adds an explicit lock
call to ensure that the string is always appended by the newline as the
users expects.
Wasn't sure if this required a test as it would be difficult since
reproducing it would be flaky.
This one might be a bit controversial since the terminology has been
introduced from the start but I think `FRACTION_LEN` is a better name
here. AFAICT it really is "the number of bits after the decimal dot when
the number is in normal form."
`MANTISSA_WIDTH` is less precise as it's unclear whether we take the
leading bit into account.
This patch also renames most of the properties to use the `_LEN` suffix
and fixes useless casts or variables.
A typo was leading to getc_unlocked.cpp.o being included into libc.a
twice.
I only noticed because I was trying to convert libc.a to a shared object
via
$ ld.lld -o libc.so --whole-archive libc.a
which errored since getc_unlocked was being defined twice.
The calculation for if a number being printed is truncated and should be
rounded up assumed a double for one of its constants, causing
occassional misrounding. This fixes that by making the constant based on
the mantissa width.
Previously, our printf would incorrectly handle conversions like
("%#x",0) and ("%#o",0). This patch corrects the behavior to match what
is described in the standard.
These bugs were found with the new printf long double fuzzing. The long
double inf vs nan bug was introduced when we changed to
get_explicit_exponent. The bitcast msan issue hadn't come up previously,
but isn't a real bug, just a poisoning confusion.
Summary:
The `fgets` function as implemented is not functional currently when
called with multiple threads. This is because we rely on reapeatedly
polling the character to detect EOF. This doesn't work when there are
multiple threads that may with to poll the characters. this patch pulls
out the logic into a standalone RPC call to handle this in a single
operation such that calling it from multiple threads functions as
expected. It also makes it less slow because we no longer make N RPC
calls for N characters.
Summary:
This function follows closely with the pattern of all the other
functions. That is, making a new opcode and forwarding the call to the
host. However, this also required modifying the test somewhat. It seems
that not all `libc` implementations follow the same error rules as are
tested here, and it is not explicit in the standard, so we simply
disable these EOF checks when targeting the GPU.
Recent testing has uncovered some hard-to-find bugs in printf's long
double support. This patch adds an extra long double path to the fuzzer
with minimal extra effort. While a more thorough long double fuzzer
would be useful, it would need to handle the non-standard cases of 80
bit long doubles such as unnormal and pseudo-denormal numbers. For that
reason, a standalone long double fuzzer is left for future development.
C++20 will automatically generate an operator== with reversed operand
order, which is ambiguous with the written operator== when one argument
is marked const and the other isn't.
This operator currently triggers -Wambiguous-reversed-operator at usage
site libc/test/UnitTest/PrintfMatcher.cpp:28.
Other libraries dependent on these libraries will automatically inherit
those compile options. This change in particular affects the compile
option "-DLIBC_COPT_STDIO_USE_SYSTEM_FILE".
printf_core.parser is not yet updated to use the printf config options. It
does not use them currently anyway and the corresponding parser_test
should be updated to respect the config options.
Summary:
This patch adds the necessary entrypoints to handle the `fseek`,
`fflush`, and `ftell` functions. These are all very straightfoward, we
simply make RPC calls to the associated function on the other end.
Implementing it this way allows us to more or less borrow the state of
the stream from the server as we intentionally maintain no internal
state on the GPU device. However, this does not implement the `errno`
functinality so that must be ignored.
When creating the new scanf reader design, I forgot to add back the
calls to flockfile and funlockfile in vfscanf_internal. This patch fixes
that, and also changes the system file version to use the normal
variants since ungetc_unlocked isn't always available.
Summary:
Normally, the implementation of `puts` simply writes a second newline
charcter after printing the first string. However, because the GPU does
everything in batches of the SIMT group size, this will end up with very
poor output where you get the strings printed and then 1-64 newline
characters all in a row. Optimizations like to turn `printf` calls into
`puts` so it's a good idea to make this produce the expected output.
The least invasive way I could do this was to add a new opcode. It's a
little bloated, but it avoids an unneccessary and slow send operation to
configure this.
In a previous patch, the printf writer was rewritten to use a single
writer class with a buffer and a callback hook. This patch refactors
scanf's reader to match conceptually.
Summary:
The parser class for stdio currently accepts different argument
providers. In-tree this is only used for a fuzzer test, however, the
proposed implementation of the GPU handling of printf / scanf will
require custom argument handlers. This makes the current approach of
using a preprocessor macro messier. This path proposed folding this
logic into a template instantiation. The downside to this is that
because the implementation of the parser class is placed into an
implementation file we need to manually instantiate the needed templates
which will slightly bloat binary size. Alternatively we could remove the
implementation file, or key off of the `libc` external packaging macro
so it is not present in the installed version.
Summary:
Previously, the `fread` operation was wrong in cases when we read less
data than was requested. That is, if we tried to read N bytes while the
file was in EOF, it would still copy N bytes of garbage. This is fixed
by only copying over the sizes we got from locally opening it rather
than just using the provided size.
Additionally, this patch simplifies the interface. The output functions
have special variants for writing to stdout / stderr. This is primarily
an optimization for these common cases so we can avoid sending the
stream as an argument which has a high delay. Because for input, we
already need to start with a `send` to tell the server how much data to
read, it costs us nothing to send the file along with it so this is
redundant. Re-use the file encoding scheme from the other
implementations, the one that stores the stream type in the LSBs of the
FILE pointer.
Summary:
This was mistakenly using the opcode for `ferror` which wasn't noticed
because tests using this weren't yet activated. This patch fixes this
mistake.
The list of printf copts available in config.json wasn't working because
the printf_core subdirectory was included before the printf_copts
variable was defined, making it effectively nothing for the printf
internals. Additionally, the tests weren't respecting the flags so they
would cause the tests to fail. This patch reorders the cmake in src and
adds flag handling in test.
Summary:
This patch implements the `fgets`, `getc`, `fgetc`, and `getchar`
functions on the GPU. Their implementations are straightforward enough.
One thing worth noting is that the implementation of `fgets` will be
extremely slow due to the high latency to read a single char. A faster
solution would be to make a new RPC call to call `fgets` (due to the
special rule that newline or null breaks the stream). But this is left
out because performance isn't the primary concern here.
The two decimal float printing styles are similar, but different in how
they end. For simplicity of writing I initially gave them different
"write_last_block" functions. This patch unifies them into one function.
Reviewed By: sivachandra
Differential Revision: https://reviews.llvm.org/D158036
This patch adds the long double table option for printf into the new
configuration scheme. This allows it to be set for most targets but
unset for baremetal.
Summary:
This patch implements fwrite, putc, putchar, and fputc on the GPU. These
are very straightforward, the main difference for the GPU implementation
is that we are currently ignoring `errno`. This patch also introduces a
minimal smoke test for `putc` that is an exact copy of the `puts` test
except we print the string char by char. This also modifies the `fopen`
test to use `fwrite` to mirror its use of `fread` so that it is tested
as well.
Summary:
The GPU uses separate implementations to perform file IO. This is all
done through the RPC interface and we kept it minimal such that we could
treat a `stdin`, `stdout`, or `stderr` handle from the CPU correctly on
the GPU. The RPC implementation uses different opcodes for whether or
not we are using one of the standard streams. This is so we do not need
to initialize anything to access the CPU's standard stream, because the
server knows that it should print to `stdout` if it gets the `STDOUT`
variant of the opcode. It also saves us an RPC call, which are expensive
relatively speaking. This patch simply cleans up this interface to make
them all use a common function. This is done in preparation to implement
some more file IO functions like getc or putc.
The %p format wasn't correctly passing along flags and modifiers to the
integer conversion behind the scenes. This patch fixes that behavior, as
well as changing the nullptr behavior to be a string conversion behind
the scenes.
Reviewed By: lntue, jhuber6
Differential Revision: https://reviews.llvm.org/D159458
Few printf config options have been setup using this new config system
along with their baremetal overrides. A follow up patch will add generation
of doc/config.rst, which will contain the full list of libc config options
and short description explaining how they affect the libc.
Reviewed By: gchatelet
Differential Revision: https://reviews.llvm.org/D159158
