Commit Graph

166 Commits

Author SHA1 Message Date
George Rimar
148022920e Recommit "rL366894: [yaml2obj] - Allow custom fields for the SHT_UNDEF sections."
With fix: do not use `stat` tool.

Original commit message:

This is a follow-up refactoring patch for recently
introduced functionality which which reduces the code duplication
and also makes possible to redefine all possible fields of
the first SHT_NULL section (previously it was only possible to set
sh_link and sh_size).

Differential revision: https://reviews.llvm.org/D65140

llvm-svn: 367003
2019-07-25 10:19:23 +00:00
JF Bastien
65217a4fa9 Revert "[yaml2obj] - Allow custom fields for the SHT_UNDEF sections."
It fails on macOS with the following error:
  https://reviews.llvm.org/D65140#1599522

llvm-svn: 366937
2019-07-24 18:29:33 +00:00
George Rimar
242da4e83e [yaml2obj] - Allow custom fields for the SHT_UNDEF sections.
This is a follow-up refactoring patch for recently
introduced functionality which which reduces the code duplication
and also makes possible to redefine all possible fields of
the first SHT_NULL section (previously it was only possible to set
sh_link and sh_size).

Differential revision: https://reviews.llvm.org/D65140

llvm-svn: 366894
2019-07-24 12:16:22 +00:00
George Rimar
ec10d5c6c1 Recommit rr366796 "[Object/ELF.h] - Improve testing of the fields in ELFFile<ELFT>::sections()."
With a fix of the issue found by UBSan.

Original commit message:

This eliminates a one error untested and
also introduces a error for one more possible case
which lead to crash previously.

Differential revision: https://reviews.llvm.org/D64987

llvm-svn: 366886
2019-07-24 11:24:37 +00:00
Vlad Tsyrklevich
aa4f8d04a9 Revert "[Object/ELF.h] - Improve testing of the fields in ELFFile<ELFT>::sections()."
This reverts commit r366796 because it was causing ubsan buildbot
failures.

llvm-svn: 366815
2019-07-23 15:02:13 +00:00
George Rimar
fccffa871c [Object/ELF.h] - Improve testing of the fields in ELFFile<ELFT>::sections().
This eliminates a one error untested and
also introduces a error for one more possible case
which lead to crash previously.

Differential revision: https://reviews.llvm.org/D64987

llvm-svn: 366796
2019-07-23 11:37:14 +00:00
George Rimar
1957d68957 [yaml2obj] - Add a support for defining null sections in YAMLs.
ELF spec shows (Figure 4-10: Section Header Table Entry:Index 0,
http://www.sco.com/developers/gabi/latest/ch4.sheader.html)
that section header at index 0 (null section) can have sh_size and
sh_link fields set to non-zero values.

It says (https://docs.oracle.com/cd/E19683-01/817-3677/6mj8mbtc9/index.html):

"If the number of sections is greater than or equal to SHN_LORESERVE (0xff00),
this member has the value zero and the actual number of section header table
entries is contained in the sh_size field of the section header at index 0.
Otherwise, the sh_size member of the initial entry contains 0."

and:

"If the section name string table section index is greater than or equal to SHN_LORESERVE
(0xff00), this member has the value SHN_XINDEX (0xffff) and the actual index of the section
name string table section is contained in the sh_link field of the section header at index 0.
Otherwise, the sh_link member of the initial entry contains 0."

At this moment it is not possible to create custom section headers at index 0 using yaml2obj.

This patch implements this.

Differential revision: https://reviews.llvm.org/D64913

llvm-svn: 366794
2019-07-23 11:03:37 +00:00
George Rimar
ab658f42a7 [yaml2elf] - Treat the SHN_UNDEF section as kind of regular section.
We have a logic that adds a few sections implicitly.
Though the SHT_NULL section with section number 0
is an exception.

In D64913 I want to teach yaml2obj to redefine the null section.
And in this patch I add it to the sections list,
to make it kind of a regular section.

Differential revision: https://reviews.llvm.org/D65087

llvm-svn: 366785
2019-07-23 07:38:44 +00:00
George Rimar
13a364e1cc [yaml2obj] - Change how we handle implicit sections.
Instead of having the special list of implicit sections,
that are mixed with the sections read from YAML on late
stages, I just create the placeholders and add them to
the main sections list early.

That allows to significantly simplify the code.

Differential revision: https://reviews.llvm.org/D64999

llvm-svn: 366677
2019-07-22 12:01:52 +00:00
George Rimar
eb41f7f081 [yaml2obj] - Allow overriding the sh_size field.
There is no way to set broken sh_size field currently
for sections. It can be usefull for writing the
test cases. 

Differential revision: https://reviews.llvm.org/D64401

llvm-svn: 365766
2019-07-11 12:59:29 +00:00
George Rimar
9df825f429 [yaml2obj] - Allow overriding sh_offset field from the YAML.
Some of our test cases are using objects which
has sections with a broken sh_offset field.

There was no way to set it from YAML until this patch.

Differential revision: https://reviews.llvm.org/D63879

llvm-svn: 364898
2019-07-02 10:20:12 +00:00
George Rimar
687d47c2b0 [yaml2obj] - Allow overriding e_shentsize, e_shoff, e_shnum and e_shstrndx fields in the YAML.
This allows setting different values for e_shentsize, e_shoff, e_shnum
and e_shstrndx fields and is useful for producing broken inputs for various
test cases.

Differential revision: https://reviews.llvm.org/D63771

llvm-svn: 364517
2019-06-27 11:08:42 +00:00
George Rimar
60dc5d4b61 [yaml2obj/obj2yaml] - Allow having the symbols and sections with duplicated names.
The patch teaches yaml2obj/obj2yaml to support parsing/dumping
the sections and symbols with the same name.
A special suffix is added to a name to make it unique.

Differential revision: https://reviews.llvm.org/D63596

llvm-svn: 364282
2019-06-25 08:22:57 +00:00
George Rimar
30ea0c4d74 [yaml2obj] - Convert ELFState<ELFT>::addSymbols method to toELFSymbols helper. NFCI.
ELFState<ELFT>::addSymbols method looks a bit strange.
User code have to create the destination symbols vector outside,
add a null symbol and then pass it to addSymbols when it seems
the more natural logic is to isolate all work with symbols inside some
function, build the list right there and return it.

Differential revision: https://reviews.llvm.org/D63493

llvm-svn: 363930
2019-06-20 14:44:48 +00:00
George Rimar
b6e20937b3 [yaml2obj/obj2yaml] - Make RawContentSection::Info Optional<>
This allows to customize this field for "implicit" sections properly.

Differential revision: https://reviews.llvm.org/D63487

llvm-svn: 363777
2019-06-19 08:57:38 +00:00
George Rimar
0aecabae14 Revert "Revert r363377: [yaml2obj] - Allow setting custom section types for implicit sections."
LLD test case will be fixed in a following commit.

Original commit message:

[yaml2obj] - Allow setting custom section types for implicit sections.

We were hardcoding the final section type for sections that
are usually implicit. The patch fixes that.

This also fixes a few issues in existent test cases and removes
one precompiled object.

Differential revision: https://reviews.llvm.org/D63267

llvm-svn: 363401
2019-06-14 14:25:34 +00:00
Rui Ueyama
9f4e21c69a Revert r363377: [yaml2obj] - Allow setting custom section types for implicit sections.
This reverts commit r363377 because lld's ELF/invalid/undefined-local-symbol-in-dso.test
test started failing after this commit.

llvm-svn: 363394
2019-06-14 13:57:25 +00:00
George Rimar
3b523c0a2e [yaml2obj] - Allow setting custom section types for implicit sections.
We were hardcoding the final section type for sections that
are usually implicit. The patch fixes that.

This also fixes a few issues in existent test cases and removes
one precompiled object.

Differential revision: https://reviews.llvm.org/D63267

llvm-svn: 363377
2019-06-14 12:16:59 +00:00
George Rimar
43f62ff17c [yaml2obj] - Allow setting the custom Address for .strtab
Despite the fact that .strtab is non-allocatable,
there is no reason to disallow setting the custom address
for it.

The patch also adds a test case showing we can set any address
we want for other implicit sections.

Differential revision: https://reviews.llvm.org/D63137

llvm-svn: 363368
2019-06-14 11:13:32 +00:00
George Rimar
cfa1a62a4c [yaml2obj] - Allow setting cutom Flags for implicit sections.
With this patch we get ability to set any flags we want
for implicit sections defined in YAML.

Differential revision: https://reviews.llvm.org/D63136

llvm-svn: 363367
2019-06-14 11:01:14 +00:00
George Rimar
ffb3c72a74 [yaml2elf] - Check we are able to set custom sh_link for .symtab/.dynsym
Allow using both custom numeric and string values for Link field of the
dynamic and regular symbol tables.

Differential revision: https://reviews.llvm.org/D63077

llvm-svn: 363042
2019-06-11 10:00:51 +00:00
George Rimar
1e41007aeb [yaml2obj/obj2yaml] - Make RawContentSection::Content and RawContentSection::Size optional
This is a follow-up for D62809.

Content and Size fields should be optional as was discussed in comments
of the D62809's thread. With that, we can describe a specific string table and
symbol table sections in a more correct way and also show appropriate errors.

The patch adds lots of test cases where the behavior is described in details.

Differential revision: https://reviews.llvm.org/D62957

llvm-svn: 362931
2019-06-10 12:43:18 +00:00
George Rimar
379aa18a39 [yaml2obj] - Do not assert when .dynsym is specified explicitly, but .dynstr is not present.
We have a code in buildSectionIndex() that adds implicit sections:

// Add special sections after input sections, if necessary.
for (StringRef Name : implicitSectionNames())
  if (SN2I.addName(Name, SecNo)) {
    // Account for this section, since it wasn't in the Doc
    ++SecNo;
    DotShStrtab.add(Name);
  }

The problem arises when .dynsym is specified explicitly and no
DynamicSymbols is used. In that case, we do not add
.dynstr implicitly and will assert later when will try to set Link
for .dynsym.

Seems, in this case, reasonable behavior is to allow Link field to be zero.
This is what this patch does.

Differential revision: https://reviews.llvm.org/D63001

llvm-svn: 362929
2019-06-10 11:38:06 +00:00
George Rimar
d71017bc86 [yaml2obj] - Remove helper methods that are probably excessive. NFC.
These methods are used only once. One of them is not used at all.

Differential revision: https://reviews.llvm.org/D63002

llvm-svn: 362925
2019-06-10 09:57:29 +00:00
George Rimar
eb394e93d2 [yaml2elf] - Refactoring followup for D62809
This is a refactoring follow-up for D62809
"Change how we handle implicit sections.".
It allows to simplify the code.

Differential revision: https://reviews.llvm.org/D62912

llvm-svn: 362777
2019-06-07 08:31:36 +00:00
George Rimar
66296dc3e4 [yaml2obj] - Change how we handle implicit sections.
We have a few sections that can be added implicitly to the output:
".dynsym", ".dynstr", ".symtab", ".strtab" and ".shstrtab".

Problem appears when such section is listed explicitly in YAML.
In that case it's content is written twice:
first time during writing of regular sections listed in the document
and second time during special handling.

Because of that their file offsets can become unexpectedly broken:
(yaml file for sample below lists .dynsym explicitly before .text.foo)

Before patch:
  [Nr] Name              Type             Address           Offset
       Size              EntSize          Flags  Link  Info  Align
  [ 0]                   NULL             0000000000000000  00000000
       0000000000000000  0000000000000000           0     0     0
  [ 1] .dynsym           DYNSYM           0000000000000100  00000250
       0000000000000030  0000000000000018   A       6     0     8
  [ 2] .text.foo         PROGBITS         0000000000000200  00000200
       0000000000000000  0000000000000000  AX       0     0     0

After patch:
Section Headers:
  [Nr] Name         Type             Address           Offset
       Size              EntSize          Flags  Link  Info  Align
  [ 0]                   NULL             0000000000000000  00000000
       0000000000000000  0000000000000000           0     0     0
  [ 1] .dynsym           DYNSYM           0000000000000100  00000200
       0000000000000030  0000000000000018   A       6     0     8
  [ 2] .text.foo         PROGBITS         0000000000000200  00000230
       0000000000000000  0000000000000000  AX       0     0     0

This patch reorganizes our code and fixes the issue described.

Differential revision: https://reviews.llvm.org/D62809

llvm-svn: 362602
2019-06-05 13:16:53 +00:00
Puyan Lotfi
a10f016006 [NFC] yaml2obj/yam2elf.cpp whitespace changes: dos2unix removed CRs.
llvm-svn: 360527
2019-05-11 17:03:36 +00:00
George Rimar
0974688a42 [yaml2obj] - Allow setting st_value explicitly for Symbol.
In some cases it is useful to explicitly set symbol's st_name value.
For example, I am using it in a patch for LLD to remove the broken
binary from a test case and replace it with a YAML test.

Differential revision: https://reviews.llvm.org/D61180

llvm-svn: 360137
2019-05-07 12:10:51 +00:00
George Rimar
366212726a [yaml2obj] - Make interface of NameToIdxMap class be human friendly and fix users.
This patch inverses the values returned by `addName` and
`lookup` methods of the class mentioned so that they
now return true on success and false on failure.
Also, it does minor code cleanup.

Differential revision: https://reviews.llvm.org/D61190

llvm-svn: 359818
2019-05-02 19:28:04 +00:00
George Rimar
f5345a3f4c [yaml2obj] - Report when unknown section is referenced from program header declaration block.
Previously we did not report this.
Also this removes multiple lookups in the map
what cleanups the code.

Differential revision: https://reviews.llvm.org/D61322

llvm-svn: 359663
2019-05-01 09:45:55 +00:00
George Rimar
bc4d3c43a3 [yaml2obj] - Simplify and reduce the code. NFC.
This inlines 2 single line static methods
and simplifies the code.

It is also possible to remove the `Is64Bit`
variable since it is used only once,
but I am not sure it will be better for readability.

llvm-svn: 359445
2019-04-29 12:25:01 +00:00
George Rimar
bd8bfd333f [yaml2obj] - Replace a loop with write_zeros(). NFCI.
This looks better.

llvm-svn: 359444
2019-04-29 12:05:53 +00:00
George Rimar
4828811a2d [yaml2obj] - Cleanup and simplify the code. NFCI.
The current code has the following problems:
`initSymtabSectionHeader` and `initStrtabSectionHeader` method
names saying us they are going to initialize the section headers.
Though for a few cases sh_flags field is initialized outside of them.
It does not look clean. This patch moves initialization of the
sh_flags inside these methods.

Also, it removes an excessive variable, what together with the above
change hopefully makes the code a bit more readable.

llvm-svn: 359443
2019-04-29 11:54:10 +00:00
George Rimar
5fcdebe75f [yaml2obj] - Make implicitSectionNames() return std::vector<StringRef>. NFCI.
No need to use SmallVector of char* here.
This simplifies the code.

llvm-svn: 359301
2019-04-26 13:09:11 +00:00
George Rimar
c1da14941f [yaml2obj] - Remove excessive variable. NFC.
`auto &Strtab` was used only once.

llvm-svn: 359300
2019-04-26 12:45:54 +00:00
George Rimar
fb7780a41f [yaml2obj] - Make the code to match the LLVM style. NFCI.
This renames the variables to uppercase and
removes use of `auto` for unobvious type.

llvm-svn: 359298
2019-04-26 12:20:51 +00:00
George Rimar
da1b3abad6 [yaml2elf] - Cleanup the initSectionHeaders(). NFCI.
This encapsulates the section specific code inside the
corresponding writeSectionContent methods.
Making the code a bit more consistent.

llvm-svn: 359297
2019-04-26 12:15:32 +00:00
George Rimar
b49e192a37 [yaml2elf] - Replace a loop with write_zeros(). NFC.
And apply clang-format to the method changed.

llvm-svn: 359090
2019-04-24 13:02:15 +00:00
George Rimar
a630b34057 [yaml2elf/obj2yaml] - Allow normal parsing/dumping of the .rela.dyn section
.rela.dyn is a section that has sh_info normally
set to zero. And Info is an optional field in the description
of the relocation section in YAML.

But currently, yaml2obj would fail to produce the object when
Info is not explicitly listed.

The patch fixes the issue.

Differential revision: https://reviews.llvm.org/D60820

llvm-svn: 358656
2019-04-18 11:02:07 +00:00
George Rimar
6da44ad75d [yaml2obj][obj2yaml] - Change how symbol's binding is descibed when parsing/dumping.
Currently, YAML has the following syntax for describing the symbols:

Symbols:
  Local:
    LocalSymbol1:
    ...
    LocalSymbol2:
    ...
  ...
  Global:
    GlobalSymbol1:
  ...
  Weak:
  ...
  GNUUnique:

I.e. symbols are grouped by their bindings. That is not very convenient,
because:

It does not allow to set a custom binding, what can be useful for producing
broken/special outputs for test cases. Adding a new binding would require to
change a syntax (what we observed when added GNUUnique recently).

It does not allow to change the order of the symbols in .symtab/.dynsym,
i.e. currently all Local symbols are placed first, then Global, Weak and GNUUnique
are following, but we are not able to change the order.

It is not consistent. Binding is just one of the properties of the symbol,
we do not group them by other properties.

It makes the code more complex that it can be. This patch shows it can be simplified
with the change performed.

The patch changes the syntax to just:

Symbols:
  Symbol1:
  ...
  Symbol2:
  ...
...

With that, we are able to work with the binding field just like with any other symbol property.

Differential revision: https://reviews.llvm.org/D60122

llvm-svn: 357595
2019-04-03 14:53:42 +00:00
George Rimar
4111299584 [yaml2obj][obj2yaml] - Teach yaml2obj/obj2yaml tools about STB_GNU_UNIQUE symbols.
yaml2obj/obj2yaml does not support the symbols with STB_GNU_UNIQUE yet.
Currently, obj2yaml fails with llvm_unreachable when met such a symbol.

I faced it when investigated the https://bugs.llvm.org/show_bug.cgi?id=41196.

Differential revision: https://reviews.llvm.org/D59875

llvm-svn: 357158
2019-03-28 10:52:14 +00:00
James Henderson
b10f48bbb4 [yaml2obj]Allow explicit setting of p_filesz, p_memsz, and p_offset
yaml2obj currently derives the p_filesz, p_memsz, and p_offset values of
program headers from their sections. This makes writing tests for
certain formats more complex, and sometimes impossible. This patch
allows setting these fields explicitly, overriding the default value,
when relevant.

Reviewed by: jakehehrlich, Higuoxing

Differential Revision: https://reviews.llvm.org/D59372

llvm-svn: 356247
2019-03-15 10:35:27 +00:00
James Henderson
9bc817a0ae [yaml2obj]Allow explicit symbol indexes in relocations and emit error for bad names
Prior to this change, the "Symbol" field of a relocation would always be
assumed to be a symbol name, and if no such symbol existed, the
relocation would reference index 0. This confused me when I tried to use
a literal symbol index in the field: since "0x1" was not a known symbol
name, the symbol index was set as 0. This change falls back to treating
unknown symbol names as integers, and emits an error if the name is not
found and the string is not an integer.

Note that the Symbol field is optional, so if a relocation doesn't
reference a symbol, it shouldn't be specified. The new error required a
number of test updates.

Reviewed by: grimar, ruiu
Differential Revision: https://reviews.llvm.org/D58510

llvm-svn: 355938
2019-03-12 17:00:25 +00:00
George Rimar
33e498b785 [yaml2obj] - Simplify. NFC.
llvm-svn: 355832
2019-03-11 16:10:02 +00:00
George Rimar
a5a0a0f049 [yaml2obj] - Allow producing ELFDATANONE ELFs
I need this to remove a binary from LLD test suite.
The patch also simplifies the code a bit.

Differential revision: https://reviews.llvm.org/D59082

llvm-svn: 355591
2019-03-07 12:09:19 +00:00
George Rimar
a7ba1a0f81 [yaml2obj] - Allow setting custom sh_info for RawContentSection sections.
This is for tweaking SHT_SYMTAB sections.
Their sh_info contains the (number of symbols + 1) usually.
But for creating invalid inputs for test cases it would be convenient
to allow explicitly override this field from YAML.

Differential revision: https://reviews.llvm.org/D58779

llvm-svn: 355193
2019-03-01 10:18:16 +00:00
James Henderson
fd99780c09 [yaml2obj]Re-allow dynamic sections to have raw content
Recently, support was added to yaml2obj to allow dynamic sections to
have a list of entries, to make it easier to write tests with dynamic
sections. However, this change also removed the ability to provide
custom contents to the dynamic section, making it hard to test
malformed contents (e.g. because the section is not a valid size to
contain an array of entries). This change reinstates this. An error is
emitted if raw content and dynamic entries are both specified.

Reviewed by: grimar, ruiu

Differential Review: https://reviews.llvm.org/D58543

llvm-svn: 354770
2019-02-25 11:02:24 +00:00
George Rimar
623ae72ad4 [yaml2obj][obj2yaml] - Support SHT_GNU_verdef (.gnu.version_d) section.
This patch adds support for parsing/dumping the .gnu.version section.

Description of the section is: https://refspecs.linuxfoundation.org/LSB_1.3.0/gLSB/gLSB/symverdefs.html

Differential revision: https://reviews.llvm.org/D58437

llvm-svn: 354574
2019-02-21 12:21:43 +00:00
George Rimar
dac37fb38d [yaml2elf] - Rename a variable. NFC.
Was suggested during review of D58441.

llvm-svn: 354463
2019-02-20 14:01:02 +00:00
George Rimar
d063c7d89b [yaml2obj] - Simplify implementation. NFCI.
Knowing about how types are declared for 32/64 bit platforms:
https://github.com/llvm-mirror/llvm/blob/master/include/llvm/BinaryFormat/ELF.h#L28

it is possible to simplify code that writes a binary a bit.
The patch does that.

Differential revision: https://reviews.llvm.org/D58441

llvm-svn: 354462
2019-02-20 13:58:43 +00:00