Commit Graph

88 Commits

Author SHA1 Message Date
Michael Jones
43c543aab7 [libc][NFC] Make strchr and strrchr more consistent
Reviewed By: sivachandra

Differential Revision: https://reviews.llvm.org/D110581
2021-09-28 17:42:03 +00:00
Andre Vieira
8b87c3d573 [libc] Add optimized memset for AArch64
Differential Revision: https://reviews.llvm.org/D107848
2021-09-23 09:19:47 +01:00
Cheng Wang
3582828748 [libc][Obvious] Some clean work with memmove. 2021-09-14 17:30:37 +08:00
Guillaume Chatelet
cc84ce9129 Revert "[libc] Some clean work with memmove."
This reverts commit b659b789c0.
2021-09-13 14:32:08 +00:00
Cheng Wang
b659b789c0 [libc] Some clean work with memmove.
- Replace `move_byte_forward()` with `memcpy`. In `memcpy` implementation,
it copies bytes forward from beginning to end. Otherwise, `memmove` unit
tests will break.
- Make `memmove` unit tests work.

Reviewed By: gchatelet

Differential Revision: https://reviews.llvm.org/D109316
2021-09-10 16:52:42 +08:00
Cheng Wang
9b015383f1 [libc][Obvious] Reorder CMakelists alphabetically. 2021-09-05 11:01:05 +08:00
Cheng Wang
7abd8f6c6e [libc][Obvious] Fix typos 2021-09-05 10:54:52 +08:00
Siva Chandra Reddy
0239adac4a [libc] Mark return value of memcpy in strcpy as initialized for msan.
Reviewed By: michaelrj

Differential Revision: https://reviews.llvm.org/D109045
2021-09-01 19:01:25 +00:00
Guillaume Chatelet
791d88f35f [libc] Align to 32B instead of 16B for optimized memcmp 2021-08-20 13:09:35 +00:00
Guillaume Chatelet
b460534ac7 [libc] Add an optimized version for memcmp
Differential Revision: https://reviews.llvm.org/D108406
2021-08-20 06:40:37 +00:00
Guillaume Chatelet
c8f79892af [libc] Add a trivial implementation for bcmp
Differential Revision: https://reviews.llvm.org/D108225
2021-08-19 17:55:16 +00:00
Guillaume Chatelet
83457d398d [libc] dedup handling of size 4 for memset 2021-08-16 22:29:10 +00:00
Guillaume Chatelet
8e4efad991 [libc] Optimize Loop strategy
Since the precondition for loop is `size >= T::kSize` we always expect
at least one run of the loop. This patch transforms the for-loop into a
do/while-loop which saves at least one test.

We also add a second template parameter to allow the Tail operation to
differ from the loop operation.
2021-08-16 22:17:56 +00:00
Guillaume Chatelet
cd2f5d5b49 [libc] rewrite aarch64 memcmp implementation
This patch is simply rearranging the code layout so it's easier to understand.

Differential Revision: https://reviews.llvm.org/D106641
2021-07-29 14:41:12 +00:00
Michael Jones
c6d03b583b [libc] add strncmp to strings
Add strncmp as a function to strings.h. Also adds unit tests, and adds
strncmp as an entrypoint for all current platforms.

Reviewed By: sivachandra

Differential Revision: https://reviews.llvm.org/D106901
2021-07-28 21:37:12 +00:00
Alfonso Gregory
0784e62c3c [libc] Fix strtok_r crash when src and *saveptr are both nullptr
While working and testing my refactoring of multiple string functions in libc, I came across a bug that needs to be addressed in a patch on its own: src is checked for nullptr and assigned to *saveptr if it is nullptr. However, saveptr is initially nullptr when it comes to reentry. This could cause a problem if both saveptr and src are null; we need to do the check first and return nullptr if both are nullptr.

Reviewed By: sivachandra

Differential Revision: https://reviews.llvm.org/D106885
2021-07-27 21:49:23 +00:00
Guillaume Chatelet
24ffb98f9d [libc] optimize bzero/memset for x86
This is simpy using the x86 optimized elements when targetting x86 cpus.

Differential Revision: https://reviews.llvm.org/D106551
2021-07-23 12:21:46 +00:00
Andre Vieira
366805ea17 [LIBC] Add an optimized memcmp implementation for AArch64
Differential Revision: https://reviews.llvm.org/D105441
2021-07-07 15:59:14 +01:00
Siva Chandra Reddy
2e9c75daff [libc] Use __builtin_ctzll instead of __builtin_ctzl in elements_x86.h.
__builtin_ctzl takes an unsigned long argument which need not be 64-bit
long on all platforms. Using __builtin_ctzll, which takes an unsigned
long long argument, ensures that 64-bit values will be handled on a
wider range of platforms.

Without this change, the test corresponding to M512 fails in Windows.

Reviewed By: gchatelet

Differential Revision: https://reviews.llvm.org/D104897
2021-06-25 22:58:13 +00:00
Guillaume Chatelet
87065c0d24 [libc] add benchmarks for memcmp and bzero
Differential Revision: https://reviews.llvm.org/D104511
2021-06-23 14:19:40 +00:00
Guillaume Chatelet
7fff39d9b0 [libc] Add a set of elementary operations
Resubmission of D100646 now making sure that we handle cases were `__builtin_memcpy_inline` is not available.

Original commit message:
Each of these elementary operations can be assembled to support higher order constructs (Overlapping access, Loop, Aligned Loop).
The patch does not compile yet as it depends on other ones (D100571, D100631) but it allows to get the conversation started.

A self-contained version of this code is available at https://godbolt.org/z/e1x6xdaxM
2021-06-16 12:11:45 +00:00
Guillaume Chatelet
c3242238b7 Revert "[libc] Add a set of elementary operations"
This reverts commit 4694321fbe.
2021-06-16 11:22:46 +00:00
Guillaume Chatelet
4694321fbe [libc] Add a set of elementary operations
Resubmission of D100646 now making sure that we handle cases were `__builtin_memcpy_inline` is not available.

Original commit message:
Each of these elementary operations can be assembled to support higher order constructs (Overlapping access, Loop, Aligned Loop).
The patch does not compile yet as it depends on other ones (D100571, D100631) but it allows to get the conversation started.

A self-contained version of this code is available at https://godbolt.org/z/e1x6xdaxM
2021-06-16 11:16:24 +00:00
Guillaume Chatelet
2e286f233e Revert "[libc] Add a set of elementary operations"
This reverts commit 8387187c2f.
2021-06-15 15:00:21 +00:00
Guillaume Chatelet
8387187c2f [libc] Add a set of elementary operations
Resubmission of D100646 now making sure that we handle cases were `__builtin_memcpy_inline` is not available.

Original commit message:
Each of these elementary operations can be assembled to support higher order constructs (Overlapping access, Loop, Aligned Loop).
The patch does not compile yet as it depends on other ones (D100571, D100631) but it allows to get the conversation started.

A self-contained version of this code is available at https://godbolt.org/z/e1x6xdaxM
2021-06-15 14:39:04 +00:00
Guillaume Chatelet
c11032ad9a Revert "[libc] Add a set of elementary operations"
This reverts commit 454d92ac3b.
2021-06-15 08:01:59 +00:00
Guillaume Chatelet
454d92ac3b [libc] Add a set of elementary operations
Resubmission of D100646 now making sure that we handle cases were `__builtin_memcpy_inline` is not available.

Original commit message:
Each of these elementary operations can be assembled to support higher order constructs (Overlapping access, Loop, Aligned Loop).
The patch does not compile yet as it depends on other ones (D100571, D100631) but it allows to get the conversation started.

A self-contained version of this code is available at https://godbolt.org/z/e1x6xdaxM
2021-06-15 07:57:13 +00:00
Guillaume Chatelet
ab45c1f21f Revert "[libc] Add a set of elementary operations"
This reverts commit e63f27a3cf.
2021-06-14 09:34:03 +00:00
Guillaume Chatelet
e63f27a3cf [libc] Add a set of elementary operations
Each of these elementary operations can be assembled to support higher order constructs (Overlapping access, Loop, Aligned Loop).
The patch does not compile yet as it depends on other ones (D100571, D100631) but it allows to get the conversation started.

Differential Revision: https://reviews.llvm.org/D100646
2021-06-14 09:01:06 +00:00
Guillaume Chatelet
6351993da7 [libc] Simplifies multi implementations
This is a roll forward of D101895 with two additional fixes:

Original Patch description:
> This is a follow up on D101524 which:
>
> - simplifies cpu features detection and usage,
> - flattens target dependent optimizations so it's obvious which implementations are generated,
> - provides an implementation targeting the host (march/mtune=native) for the mem* functions,
> - makes sure all implementations are unittested (provided the host can run them).

Additional fixes:
 - Fix uninitialized ALL_CPU_FEATURES
 - Use non pseudo microarch as it is only supported from Clang 12 on

Differential Revision: https://reviews.llvm.org/D102233
2021-05-12 07:24:53 +00:00
Siva Chandra Reddy
0c64cef894 [libc] Rever "Simplifies multi implementations and benchmarks".
This reverts commit 541f107871 as the bots
are failing with unknown architecture "x86-64-v*". Will let the original
author decide on the right course of action to correct the problem and
reland.
2021-05-10 19:20:27 +00:00
Guillaume Chatelet
541f107871 [libc] Simplifies multi implementations and benchmarks
This is a follow up on D101524 which:
 - simplifies cpu features detection and usage,
 - flattens target dependent optimizations so it's obvious which implementations are generated,
 - provides an implementation targeting the host (march/mtune=native) for the mem* functions,
 - makes sure all implementations are unittested (provided the host can run them),
 - makes sure all implementations are benchmarkable (provided the host can run them).

Differential Revision: https://reviews.llvm.org/D101895
2021-05-10 08:23:30 +00:00
Guillaume Chatelet
7c2ece523d [libc] Normalize LIBC_TARGET_MACHINE
Current implementation defines LIBC_TARGET_MACHINE with the use of CMAKE_SYSTEM_PROCESSOR.
Unfortunately CMAKE_SYSTEM_PROCESSOR is OS dependent and can produce different results.
An evidence of this is the various matchers used to detect whether the architecture is x86.

This patch normalizes LIBC_TARGET_MACHINE and renames it LIBC_TARGET_ARCHITECTURE.
I've added many architectures but we may want to limit ourselves to x86 and ARM.

Differential Revision: https://reviews.llvm.org/D101524
2021-05-05 15:52:42 +00:00
Guillaume Chatelet
b5f04d81a2 [libc] Use different alignment for memcpy between ARM and x86.
Aligned copy used to be 'destination aligned' for x86 but this decision was reverted in D93457 where we noticed that it was better for ARM to be 'source aligned'.
More benchmarking confirmed that it can be up to 30% faster to align copy to destination for x86. This Patch offers both implementations and switches x86 back to destination aligned.
It also fixes alignment to 32 byte on x86.

Differential Revision: https://reviews.llvm.org/D101296
2021-04-26 19:30:00 +00:00
Guillaume Chatelet
77d81c2270 [libc] Fix msan/asan memcpy reentrancy
This is needed to prevent asan/msan instrumentation to redirect CopyBlock to `__asan_memcpy` (resp. `__msan_memcpy`).
These functions would then differ operation to `memcpy` which leads to reentrancy issues.

With this patch, `memcpy` is fully instrumented and covered by asan/msan.

If this turns out to be too expensive, instrumentation can be selectively or fully disabled through the use of the `__attribute__((no_sanitize(address, memory)))` annotation.

Differential Revision: https://reviews.llvm.org/D99598
2021-03-30 15:28:47 +00:00
Siva Chandra Reddy
b937908c37 [libc][NFC] Move the template implementation of integer_abs to __support.
This eliminates cross-header dependency from stdlib to string.
2021-03-11 20:20:18 -08:00
Siva Chandra Reddy
5f0800cc18 [libc][NFC] Remove headergen for the cacheline size macro.
We want to be able to build and test the string functions in contexts
like that of Fuchsia where LLVM pieces like tablegen are not available.
Since header generation uses tablegen, we are removing the dependency on
headergen here.

Reviewed By: gchatelet

Differential Revision: https://reviews.llvm.org/D97363
2021-02-24 07:29:09 -08:00
Andre Vieira
369f7de313 [LIBC] Add optimized memcpy routine for AArch64
This patch adds an optimized memcpy routine for AArch64 tuned and benchmarked
on Neoverse-N1.

Differential Revision: https://reviews.llvm.org/D92235
2021-02-03 09:30:55 +00:00
Cheng Wang
83bd242202 [libc][Obvious] Remove DEPS for unistd.h in CMake file of memmove. 2021-01-29 17:05:24 +08:00
Guillaume Chatelet
e517dff50a [libc][NFC] remove dependency on non standard ssize_t
`ssize_t` is from POSIX and is not standard unfortunately.
Rewritting the code so it doesn't depend on it.

Differential Revision: https://reviews.llvm.org/D94760
2021-01-19 08:12:38 +00:00
Guillaume Chatelet
5bf47e142b [libc] CopyAlignedBlocks can now specify alignment on top of block size
This has been requested in D92236

Differential Revision: https://reviews.llvm.org/D94770
2021-01-15 15:32:02 +00:00
Guillaume Chatelet
a10300a2b2 [libc] Allow customization of memcpy via flags.
- Adds LLVM_LIBC_IS_DEFINED macro to libc/src/__support/common.h
 - Adds a few knobs to memcpy to help with experimentations:
   - LLVM_LIBC_MEMCPY_X86_USE_ONLY_REPMOVSB replaces the implementation with a single call to rep;movsb
   - LLVM_LIBC_MEMCPY_X86_USE_REPMOVSB_FROM_SIZE customizes where the usage of rep;movsb

Differential Revision: https://reviews.llvm.org/D94692
2021-01-15 09:26:45 +00:00
Cheng Wang
2423ec5837 [libc] Add memmove implementation.
Use `memcpy` rather than copying bytes one by one, for there might be large
size structs to move.

Reviewed By: gchatelet, sivachandra

Differential Revision: https://reviews.llvm.org/D93195
2021-01-15 12:08:25 +08:00
Michael Jones
a0b65a7bcd [libc] Switch to use a macro which does not insert a section for every libc function.
Summary:
The new macro also inserts the C alias for the C++ implementations
without needing an objcopy based post processing step. The CMake
rules have been updated to reflect this. More CMake cleanup can be
taken up in future rounds and appropriate TODOs have been added for them.

Reviewers: mcgrathr, sivachandra

Subscribers:
2021-01-08 23:52:35 +00:00
Guillaume Chatelet
aa9db51ef6 [libc] Align src buffer instead of dst buffer
We used to align destination buffer instead of source buffer for the loop of block copy.
This is a mistake.

Differential Revision: https://reviews.llvm.org/D93457
2021-01-06 12:04:53 +00:00
Cheng Wang
af68c3b892 [libc] Add memcmp implementation.
Reviewed By: gchatelet

Differential Revision: https://reviews.llvm.org/D93009
2020-12-15 09:47:29 +08:00
Cheng Wang
60cef89362 [libc] Add strncpy implementation.
Add libc strncpy implementation.

Reviewed By: sivachandra, gchatelet

Differential Revision: https://reviews.llvm.org/D91399
2020-12-02 20:45:51 +08:00
Guillaume Chatelet
699d17d4d6 [libc] Improve memcpy copy loop
Rewriting loop so the terminating condition does not depend on the loop body

Differential Revision: https://reviews.llvm.org/D91976
2020-11-30 08:24:10 +00:00
Michael Jones
3e18fb3390 [libc] Switch functions to using global headers
This switches all of the files in src/string, src/math, and
test/src/math from using relative paths (e.g. `#include “include/string.h”`)
to global paths (e.g. `#include <string.h>`) to make bringing up those
functions on other platforms, such as fuchsia, easier.

Reviewed By: sivachandra

Differential Revision: https://reviews.llvm.org/D91394
2020-11-21 00:07:17 +00:00
Guillaume Chatelet
c3fd2a50ba [libc] Remove special case for 8 and 16 bytes
They don't seem to gain much in real apps and its better to favor less branches and smaller code.
2020-09-15 20:48:27 +00:00