Commit Graph

169 Commits

Author SHA1 Message Date
Andre Vieira
366805ea17 [LIBC] Add an optimized memcmp implementation for AArch64
Differential Revision: https://reviews.llvm.org/D105441
2021-07-07 15:59:14 +01:00
Siva Chandra Reddy
804dc3dcf2 [libc] Clear all exceptions before setting in fesetexceptflag.
Previously, exceptions from the flag were being added. This patch
changes it such that only the exceptions in the flag will be set.

Reviewed By: lntue

Differential Revision: https://reviews.llvm.org/D105085
2021-06-30 17:29:48 +00:00
Siva Chandra Reddy
2e9c75daff [libc] Use __builtin_ctzll instead of __builtin_ctzl in elements_x86.h.
__builtin_ctzl takes an unsigned long argument which need not be 64-bit
long on all platforms. Using __builtin_ctzll, which takes an unsigned
long long argument, ensures that 64-bit values will be handled on a
wider range of platforms.

Without this change, the test corresponding to M512 fails in Windows.

Reviewed By: gchatelet

Differential Revision: https://reviews.llvm.org/D104897
2021-06-25 22:58:13 +00:00
Guillaume Chatelet
87065c0d24 [libc] add benchmarks for memcmp and bzero
Differential Revision: https://reviews.llvm.org/D104511
2021-06-23 14:19:40 +00:00
Guillaume Chatelet
7fff39d9b0 [libc] Add a set of elementary operations
Resubmission of D100646 now making sure that we handle cases were `__builtin_memcpy_inline` is not available.

Original commit message:
Each of these elementary operations can be assembled to support higher order constructs (Overlapping access, Loop, Aligned Loop).
The patch does not compile yet as it depends on other ones (D100571, D100631) but it allows to get the conversation started.

A self-contained version of this code is available at https://godbolt.org/z/e1x6xdaxM
2021-06-16 12:11:45 +00:00
Guillaume Chatelet
c3242238b7 Revert "[libc] Add a set of elementary operations"
This reverts commit 4694321fbe.
2021-06-16 11:22:46 +00:00
Guillaume Chatelet
4694321fbe [libc] Add a set of elementary operations
Resubmission of D100646 now making sure that we handle cases were `__builtin_memcpy_inline` is not available.

Original commit message:
Each of these elementary operations can be assembled to support higher order constructs (Overlapping access, Loop, Aligned Loop).
The patch does not compile yet as it depends on other ones (D100571, D100631) but it allows to get the conversation started.

A self-contained version of this code is available at https://godbolt.org/z/e1x6xdaxM
2021-06-16 11:16:24 +00:00
Guillaume Chatelet
2e286f233e Revert "[libc] Add a set of elementary operations"
This reverts commit 8387187c2f.
2021-06-15 15:00:21 +00:00
Guillaume Chatelet
8387187c2f [libc] Add a set of elementary operations
Resubmission of D100646 now making sure that we handle cases were `__builtin_memcpy_inline` is not available.

Original commit message:
Each of these elementary operations can be assembled to support higher order constructs (Overlapping access, Loop, Aligned Loop).
The patch does not compile yet as it depends on other ones (D100571, D100631) but it allows to get the conversation started.

A self-contained version of this code is available at https://godbolt.org/z/e1x6xdaxM
2021-06-15 14:39:04 +00:00
Guillaume Chatelet
c11032ad9a Revert "[libc] Add a set of elementary operations"
This reverts commit 454d92ac3b.
2021-06-15 08:01:59 +00:00
Guillaume Chatelet
454d92ac3b [libc] Add a set of elementary operations
Resubmission of D100646 now making sure that we handle cases were `__builtin_memcpy_inline` is not available.

Original commit message:
Each of these elementary operations can be assembled to support higher order constructs (Overlapping access, Loop, Aligned Loop).
The patch does not compile yet as it depends on other ones (D100571, D100631) but it allows to get the conversation started.

A self-contained version of this code is available at https://godbolt.org/z/e1x6xdaxM
2021-06-15 07:57:13 +00:00
Siva Chandra Reddy
a58b2827fe [libc] Add hardware implementations of x86_64 sqrt functions. 2021-06-14 21:25:37 +00:00
Guillaume Chatelet
ab45c1f21f Revert "[libc] Add a set of elementary operations"
This reverts commit e63f27a3cf.
2021-06-14 09:34:03 +00:00
Guillaume Chatelet
e63f27a3cf [libc] Add a set of elementary operations
Each of these elementary operations can be assembled to support higher order constructs (Overlapping access, Loop, Aligned Loop).
The patch does not compile yet as it depends on other ones (D100571, D100631) but it allows to get the conversation started.

Differential Revision: https://reviews.llvm.org/D100646
2021-06-14 09:01:06 +00:00
Tue Ly
4e5f8b4d8d [libc] Add implementation of expm1f.
Use expm1f(x) = exp(x) - 1 for |x| > ln(2).
For |x| <= ln(2), divide it into 3 subintervals: [-ln2, -1/8], [-1/8, 1/8], [1/8, ln2]
and use a degree-6 polynomial approximation generated by Sollya's fpminmax for each interval.
Errors < 1.5 ULPs when we use fma to evaluate the polynomials.

Differential Revision: https://reviews.llvm.org/D101134
2021-06-10 14:58:34 -04:00
Siva Chandra Reddy
7deb5ef44f [libc][NFC] Instead of erroring, skip math targets with missing implementations.
Fixes Aarch64 bot.
2021-05-13 19:22:11 +00:00
Siva Chandra Reddy
861dc75906 [libc] Add x86_64 implementations of double precision cos, sin and tan.
The implementations use the x86_64 FPU instructions. These instructions
are extremely slow compared to a polynomial based software
implementation. Also, their accuracy falls drastically once the input
goes beyond 2PI. To improve both the speed and accuracy, we will be
taking the following approach going forward:
1. As a follow up to this CL, we will implement a range reduction algorithm
which will expand the accuracy to the entire double precision range.
2. After that, we will replace the HW instructions with a polynomial
implementation to improve the run time.

After step 2, the implementations will be accurate, performant and target
architecture independent.

Reviewed By: lntue

Differential Revision: https://reviews.llvm.org/D102384
2021-05-13 19:02:00 +00:00
Guillaume Chatelet
6351993da7 [libc] Simplifies multi implementations
This is a roll forward of D101895 with two additional fixes:

Original Patch description:
> This is a follow up on D101524 which:
>
> - simplifies cpu features detection and usage,
> - flattens target dependent optimizations so it's obvious which implementations are generated,
> - provides an implementation targeting the host (march/mtune=native) for the mem* functions,
> - makes sure all implementations are unittested (provided the host can run them).

Additional fixes:
 - Fix uninitialized ALL_CPU_FEATURES
 - Use non pseudo microarch as it is only supported from Clang 12 on

Differential Revision: https://reviews.llvm.org/D102233
2021-05-12 07:24:53 +00:00
Siva Chandra Reddy
0c64cef894 [libc] Rever "Simplifies multi implementations and benchmarks".
This reverts commit 541f107871 as the bots
are failing with unknown architecture "x86-64-v*". Will let the original
author decide on the right course of action to correct the problem and
reland.
2021-05-10 19:20:27 +00:00
Guillaume Chatelet
541f107871 [libc] Simplifies multi implementations and benchmarks
This is a follow up on D101524 which:
 - simplifies cpu features detection and usage,
 - flattens target dependent optimizations so it's obvious which implementations are generated,
 - provides an implementation targeting the host (march/mtune=native) for the mem* functions,
 - makes sure all implementations are unittested (provided the host can run them),
 - makes sure all implementations are benchmarkable (provided the host can run them).

Differential Revision: https://reviews.llvm.org/D101895
2021-05-10 08:23:30 +00:00
Guillaume Chatelet
7c2ece523d [libc] Normalize LIBC_TARGET_MACHINE
Current implementation defines LIBC_TARGET_MACHINE with the use of CMAKE_SYSTEM_PROCESSOR.
Unfortunately CMAKE_SYSTEM_PROCESSOR is OS dependent and can produce different results.
An evidence of this is the various matchers used to detect whether the architecture is x86.

This patch normalizes LIBC_TARGET_MACHINE and renames it LIBC_TARGET_ARCHITECTURE.
I've added many architectures but we may want to limit ourselves to x86 and ARM.

Differential Revision: https://reviews.llvm.org/D101524
2021-05-05 15:52:42 +00:00
Raman Tenneti
a72499e475 [libc] Introduce asctime, asctime_r to LLVM libc
[libc] Introduce asctime, asctime_r to LLVM libc

asctime and asctime_r share the same common code. They call asctime_internal
a static inline function.

asctime uses snprintf to return the string representation in a buffer.
It uses the following format (26 characters is the buffer size) as per
7.27.3.1 section in http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2478.pdf.
The buf parameter for asctime_r shall point to a buffer of at least 26 bytes.

snprintf(buf, 26, "%.3s %.3s%3d %.2d:%.2d:%.2d %d\n",...)

Reviewed By: sivachandra

Differential Revision: https://reviews.llvm.org/D99686
2021-05-03 17:15:00 -07:00
Guillaume Chatelet
b5f04d81a2 [libc] Use different alignment for memcpy between ARM and x86.
Aligned copy used to be 'destination aligned' for x86 but this decision was reverted in D93457 where we noticed that it was better for ARM to be 'source aligned'.
More benchmarking confirmed that it can be up to 30% faster to align copy to destination for x86. This Patch offers both implementations and switches x86 back to destination aligned.
It also fixes alignment to 32 byte on x86.

Differential Revision: https://reviews.llvm.org/D101296
2021-04-26 19:30:00 +00:00
Siva Chandra
95934c3a37 [libc] Add hardware implementations of fma and fmaf for x86_64 and aarch64.
The current generic implementation of the fmaf function has been moved
to the FPUtil directory. This allows one use the fma operation from
implementations of other math functions like the trignometric functions
without depending on/requiring the fma/fmaf/fmal function targets. If
this pattern ends being convenient, we will switch all generic math
implementations to this pattern.

Reviewed By: lntue

Differential Revision: https://reviews.llvm.org/D100811
2021-04-21 04:31:27 +00:00
Guillaume Chatelet
7e075ad0b2 [libc] Add endianness support
Add endianness detection support. This will be useful to implement `memcmp`.

Differential Revision: https://reviews.llvm.org/D100571
2021-04-16 21:35:08 +00:00
Siva Chandra Reddy
6666e0d7a2 [libc] Make FPBits a union.
This helps us avoid the uncomfortable reinterpret-casts. Avoiding the
reinterpret casts prevents us from tripping the sanitizers as well.

Reviewed By: lntue

Differential Revision: https://reviews.llvm.org/D100360
2021-04-13 09:21:35 -07:00
Guillaume Chatelet
77d81c2270 [libc] Fix msan/asan memcpy reentrancy
This is needed to prevent asan/msan instrumentation to redirect CopyBlock to `__asan_memcpy` (resp. `__msan_memcpy`).
These functions would then differ operation to `memcpy` which leads to reentrancy issues.

With this patch, `memcpy` is fully instrumented and covered by asan/msan.

If this turns out to be too expensive, instrumentation can be selectively or fully disabled through the use of the `__attribute__((no_sanitize(address, memory)))` annotation.

Differential Revision: https://reviews.llvm.org/D99598
2021-03-30 15:28:47 +00:00
Raman Tenneti
8b35159ac7 [libc] Introduces gmtime_r to LLVM libc, based on C99/C2X/Single Unix Sp.
gmtime and gmtime_r share the same common code. They call gmtime_internal
a static inline function. Thus added only validation tests for gmtime_r.

Reviewed By: sivachandra

Differential Revision: https://reviews.llvm.org/D99046
2021-03-22 14:04:05 -07:00
Raman Tenneti
eaae52c1fd This introduces gmtime to LLVM libc, based on C99/C2X/Single Unix Spec.
This change doesn't handle TIMEZONE, tm_isdst and leap seconds.

Moved shared code between mktime and gmtime into time_utils.cpp.

Reviewed By: sivachandra

Differential Revision: https://reviews.llvm.org/D98467
2021-03-16 16:44:48 -07:00
Siva Chandra Reddy
7c0179129f [libc][Obvious] Fix except flags reading overflow detected by asan. 2021-03-13 16:30:33 -08:00
Siva Chandra Reddy
e9e788d145 [libc] Introduce a full build mode CMake option.
This option will build LLVM libc as a full libc by itself. In this mode,
it is not expected that it will be mixed with other libcs. The
non-full-build mode will be the default LLVM libc build mode. In a future
where LLVM libc is complete enough, the full libc build will be made the
default mode.
2021-03-12 13:28:40 -08:00
Siva Chandra Reddy
b937908c37 [libc][NFC] Move the template implementation of integer_abs to __support.
This eliminates cross-header dependency from stdlib to string.
2021-03-11 20:20:18 -08:00
Siva Chandra Reddy
3f3f88fb95 [libc][Obvious] Add a file which was missed in 001a12ed59. 2021-03-08 15:39:29 -08:00
Siva Chandra Reddy
001a12ed59 [libc][NFC] Make x86_64 fenv functions msan safe.
These functions used inline asm to read FPU state. This change adds
explicit unpoisoning in these functions as the sanitizers don't see the
read operations.
2021-03-08 15:20:08 -08:00
Siva Chandra Reddy
9550f8ba9a [libc][NFC] Make few fenv functions work with fexcept_t from other libcs. 2021-03-04 11:25:28 -08:00
Siva Chandra Reddy
35e2e448ce [libc] Remove redundant header files included from internal paths. 2021-03-03 21:29:46 -08:00
Siva Chandra Reddy
5f0800cc18 [libc][NFC] Remove headergen for the cacheline size macro.
We want to be able to build and test the string functions in contexts
like that of Fuchsia where LLVM pieces like tablegen are not available.
Since header generation uses tablegen, we are removing the dependency on
headergen here.

Reviewed By: gchatelet

Differential Revision: https://reviews.llvm.org/D97363
2021-02-24 07:29:09 -08:00
Raman Tenneti
80bea4a0d5 [libc] [Obvious] Fix. 2021-02-22 19:15:35 -08:00
Raman Tenneti
034f562925 Changes to mktime to handle invalid dates, overflow and underflow andcalculating the correct date and thenumber of seconds even if invalid datesare passed as arguments.
Added tests for invalid dates like the following
  Date 1970-01-01 00:00:-1 is treated as 1969-12-31 23:59:59 and seconds
  are returned for the modified date.

Tested the code by doing ninja check-libc (and cmake).

Reviewed By: sivachandra, rtenneti

Differential Revision: https://reviews.llvm.org/D96684
2021-02-22 18:37:40 -08:00
Siva Chandra Reddy
b7e05c874b [libc] Add implementations of the remaining fenv functions.
Namely, implementations of fegetexceptfflag, fesetexceptflag,
fegetenv, fesetenv, feholdexcept and feupdateenv have been added.

Reviewed By: lntue

Differential Revision: https://reviews.llvm.org/D96935
2021-02-18 13:29:40 -08:00
Siva Chandra
c90c8d38d3 [libc] Add aarch64 flavors of floor, round, sqrt and trunc.
Only single and double precision flavors have been added.

Reviewed By: lntue, sdesmalen

Differential Revision: https://reviews.llvm.org/D95999
2021-02-05 10:41:32 -08:00
Siva Chandra
bbb7555403 [libc][NFC] Move generic math implementations to the generic directory.
This expands the pattern suggest in https://reviews.llvm.org/D95850 to all
math functions.
2021-02-03 10:46:00 -08:00
Andre Vieira
369f7de313 [LIBC] Add optimized memcpy routine for AArch64
This patch adds an optimized memcpy routine for AArch64 tuned and benchmarked
on Neoverse-N1.

Differential Revision: https://reviews.llvm.org/D92235
2021-02-03 09:30:55 +00:00
Siva Chandra
2668714747 [libc] Add hardware implementations of ceil and ceilf for aarch64.
This change also introduces a new source layout for adding machine
specific and generic implementations. To keep the scope of this change
small, this new pattern is only applied for ceil, ceilf and ceill.
Follow up changes will switch all math functions in to the new pattern.

Reviewed By: lntue

Differential Revision: https://reviews.llvm.org/D95850
2021-02-02 15:36:19 -08:00
Cheng Wang
83bd242202 [libc][Obvious] Remove DEPS for unistd.h in CMake file of memmove. 2021-01-29 17:05:24 +08:00
Guillaume Chatelet
e517dff50a [libc][NFC] remove dependency on non standard ssize_t
`ssize_t` is from POSIX and is not standard unfortunately.
Rewritting the code so it doesn't depend on it.

Differential Revision: https://reviews.llvm.org/D94760
2021-01-19 08:12:38 +00:00
Guillaume Chatelet
5bf47e142b [libc] CopyAlignedBlocks can now specify alignment on top of block size
This has been requested in D92236

Differential Revision: https://reviews.llvm.org/D94770
2021-01-15 15:32:02 +00:00
Guillaume Chatelet
a10300a2b2 [libc] Allow customization of memcpy via flags.
- Adds LLVM_LIBC_IS_DEFINED macro to libc/src/__support/common.h
 - Adds a few knobs to memcpy to help with experimentations:
   - LLVM_LIBC_MEMCPY_X86_USE_ONLY_REPMOVSB replaces the implementation with a single call to rep;movsb
   - LLVM_LIBC_MEMCPY_X86_USE_REPMOVSB_FROM_SIZE customizes where the usage of rep;movsb

Differential Revision: https://reviews.llvm.org/D94692
2021-01-15 09:26:45 +00:00
Cheng Wang
2423ec5837 [libc] Add memmove implementation.
Use `memcpy` rather than copying bytes one by one, for there might be large
size structs to move.

Reviewed By: gchatelet, sivachandra

Differential Revision: https://reviews.llvm.org/D93195
2021-01-15 12:08:25 +08:00
Roland McGrath
e7228062b2 [libc] Use #undef isascii in specific header
Standard C allows all standard headers to declare macros for all
their functions.  So after possibly including any standard header
like <ctype.h>, it's perfectly normal for any and all of the
functions it declares to be defined as macros.  Standard C requires
explicit `#undef` before using that identifier in a way that is not
compatible with function-like macro definitions.

The C standard's rules for this are extended to POSIX as well for
the interfaces it defines, and it's the expected norm for
nonstandard extensions declared by standard C library headers too.

So far the only place this has come up for llvm-libc's code is with
the isascii function in Fuchsia's libc.  But other cases can arise
for any standard (or common extension) function names that source
code in llvm-libc is using in nonstandard ways, i.e. as C++
identifiers.

The only correct and robust way to handle the possible inclusion of
standard C library headers when building llvm-libc source code is to
use `#undef` explicitly for each identifier before using it.  The
easy and obvious place to do that is in the per-function header.
This requires that all code, such as test code, that might include
any standard C library headers, e.g. via utils/UnitTest/Test.h, make
sure to include those *first* before the per-function header.

This change does that for isascii and its test.  But it should be
done uniformly for all the code and documented as a consistent
convention so new implementation files are sure to get this right.

Reviewed By: sivachandra

Differential Revision: https://reviews.llvm.org/D94642
2021-01-14 13:25:05 -08:00