Update VPlan to include the scalar loop header. This allows retiring
VPLiveOut, as the remaining live-outs can now be handled by adding
operands to the wrapped phis in the scalar loop header.
Note that the current version only includes the scalar loop header, no
other loop blocks and also does not wrap it in a region block.
PR: https://github.com/llvm/llvm-project/pull/109975
Use getVectorLoopRegion to retrieve the vector loop header instead of
relying on getting the successor of the entry block. This makes sure the
code still works correctly as more parts of the skeleton are modeled in
VPlan.
This patch implements a ready-list-based scheduler that operates on
DependencyGraph.
It is used by the sandbox vectorizer to test the legality of vectorizing
a group of instrs.
SchedBundle is a helper container, containing all DGNodes that
correspond to the instructions that we are attempting to schedule with
trySchedule(Instrs).
This allows calling ::dump() on various sub-classes of VPSingleDefRecipe
directly, as it resolves an ambigous name lookup.
Previously, calling VPWidenRecipe::dump() (and others), would result in
the following errors:
llvm/unittests/Transforms/Vectorize/VPlanTest.cpp:1284:19: error: member 'dump' found in multiple base classes of different types
1284 | WidenR->dump();
| ^
llvm/include/../lib/Transforms/Vectorize/VPlanValue.h:434:8: note: member found by ambiguous name lookup
434 | void dump() const;
| ^
llvm/include/../lib/Transforms/Vectorize/VPlanValue.h:108:8: note: member found by ambiguous name lookup
108 | void dump() const;
| ^
1 error generated.
This patch implements the UnscheduledSuccs counter in DGNode. It counts
the number of unscheduled successors and is used by the scheduler to
determine when a node is ready.
This patch implements `Interval::comesBefore(const Interval &Other)`
which returns true if this interval is strictly before Other in program
order. The function asserts that the intervals are disjoint.
Rename the function to reflect its correct behavior and to be consistent
with `Module::getOrInsertFunction`. This is also in preparation of
adding a new `Intrinsic::getDeclaration` that will have behavior similar
to `Module::getFunction` (i.e, just lookup, no creation).
This patch splits off intrinsic hanlding to a new
VPWidenIntrinsicRecipe. VPWidenIntrinsicRecipes only need access to the
intrinsic ID to widen and the scalar result type (in case the intrinsic
is overloaded on the result type). It does not need access to an
underlying IR call instruction or function.
This means VPWidenIntrinsicRecipe can be created easily without access
to underlying IR.
[Retry 110696 with a proper rebase.]
Seed collection will assemble instructions to be vectorized into
SeedBundles. This data structure is not intended to be used directly,
but will be the basis for load bundles, store bundles, and so on.
This patch moves:
- Utils::isStackSaveOrRestoreIntrinsic()
- Utils::isMemIntrinsic()
- Utils::isMemDepCandidate()
to DGNode because they no longer require LLVM IR access and are used
only by the DAG.
This patch implements the MemDGNode class for DAG nodes that are
candidates
for memory dependencies. These nodes form a chain that is accessible by
`getPrevNode()` and `getNextNode()`.
It also implements a builder class that creates MemDGNode intervals from
Instructions.
This patch implements a few set operations for the intervals. These
include:
- operator==() and operator!=() for comparing two intervals.
- disjoint()
- intersection()
- difference, which uses operator-()
This patch converts InstrInterval class to a class template and renames
InstrInterval to Itnerval.
This change will allow us to reuse the Interval for dependency graph
nodes.
I'm planning to add RegionPass and RegionPassManager next to the
equivalent FunctionPass and FunctionPassManager in SandboxIR, which
means that SandboxIR has to know about Regions.
There's nothing vectorizer-specific about the Region class, and the only
thing using Regions at this moment is the unit test, so this is a
straightforward file move.
Unify logic for mayWriteToMemory and mayHaveSideEffects for
VPInstruction, with the later relying on the former. Also extend to
handle binary operators.
Split off from https://github.com/llvm/llvm-project/pull/106441
For each region, we create a metadata node. Then when an instruction is
added to the Region, it gets tagged with the metadata node for that
region. In the following example, we have a Region that contains only
the `%t0` instruction.
```
define i8 @foo(i8 %v0, i8 %v1) {
%t0 = add i8 %v0, 1, !sbvec !0
%t1 = add i8 %t0, %v1
ret i8 %t1
}
!0 = distinct !{!"region"}
```
This commit also adds a function to create regions from metadata already
present in a Function.
This metadata can be used for debugging: if we dump IR before a Region
pass, the IR will contain enough info to re-create the Region and run
the pass by itself in a later invocation.
---------
Co-authored-by: Alina Sbirlea <alina.g.simion@gmail.com>
