Jeffrey Byrnes
c9a87a50ae
[SLPVectorizer] Use accurate cost for external users of resize shuffles ( #137419 )
...
When implementing the vectorization, we potentially need to add shuffles
for external users. In such cases, we may be shuffling a smaller vector
into a larger vector. When this happens `ResizeToVF` will just build a
poison padded identity vector. Then the to build the final shuffle, we
just use the `SK_InsertSubvector` mask.
This is possibly clearer by looking at the included test in
SLPVectorizer/AMDGPU/external-shuffle.ll
In the exit block we have a bunch of shuffles to glue the vectorized
tree match the `InsertElement` users. `TMP25` holds the result of
resizing the v2i16 vectorized sequence to match the `InsertElement` size
v16i16. Then `TMP26` is the final shuffle which replaces the
`InsertElement` sequence. This is just an insertsubvector.
However, when calculating the cost for these shuffles, we aren't
modelling this correctly. `ResizeToVF` will indicate to
`performExtractsShuffleAction` that we cannot use the original mask due
to the resize shuffle. The consequence is that the cost calculation uses
a different shuffle mask than what is ultimately used.
Going back to the included test, we can consider again `TMP26`. Clearly
we can see the shuffle uses a mask {0, 1, 2, 3, 16, 17, poison ..}.
However, we will currently calculate the cost with a mask {0, 1, 2, 3,
20, 21, ...} we have replaced 16 and 17 with 20 and 21 (Index + Vector
Size). Queries like BasicTTImpl::improveShuffleKindFromMask will not
recognize this as an `SK_InsertSubvector` mask, and targets which have
reduced costs for `SK_InsertSubvector` will not accurately calculate the
cost.
2025-06-17 08:14:05 -07:00
..
2025-06-13 18:25:07 +01:00
2025-06-13 18:25:07 +01:00
2025-03-12 08:18:51 -07:00
2025-03-12 08:18:51 -07:00
2024-11-22 06:50:25 -08:00
2025-03-12 08:18:51 -07:00
2025-03-12 08:18:51 -07:00
2025-03-12 08:18:51 -07:00
2025-03-12 08:18:51 -07:00
2024-11-06 11:53:33 +00:00
2024-11-06 11:53:33 +00:00
2025-01-24 15:13:13 +00:00
2025-01-24 09:47:40 +00:00
2025-01-24 15:13:13 +00:00
2025-01-24 09:47:40 +00:00
2024-11-29 06:51:19 -08:00
2025-04-16 00:08:59 +08:00
2025-04-16 00:08:59 +08:00
2025-02-14 11:03:33 -08:00
2025-04-16 00:08:59 +08:00
2025-04-16 00:08:59 +08:00
2025-02-26 13:25:37 -08:00
2025-03-12 08:18:51 -07:00
2024-11-11 10:49:39 -08:00
2025-03-24 09:29:18 -04:00
2025-05-17 11:06:37 -07:00
2025-03-06 10:12:49 -08:00
2025-02-24 06:48:43 -08:00
2024-11-19 11:31:40 -08:00
2025-04-16 00:08:59 +08:00
2024-11-06 11:53:33 +00:00
2025-02-04 19:09:57 -05:00
2025-04-01 14:30:06 -04:00
2025-04-17 09:10:15 -07:00
2024-11-06 11:53:33 +00:00
2025-02-21 07:15:02 -05:00
2025-03-31 10:52:12 -07:00
2024-11-06 11:53:33 +00:00
2024-11-06 11:53:33 +00:00
2024-10-29 17:16:17 -07:00
2025-02-19 09:43:26 -08:00
2025-05-18 12:59:36 -07:00
2024-12-03 20:54:36 +00:00
2025-02-04 18:49:43 -05:00
2024-11-06 11:53:33 +00:00
2024-12-03 20:54:36 +00:00
2024-12-03 20:54:36 +00:00
2024-12-03 20:54:36 +00:00
2024-12-03 20:54:36 +00:00
2024-12-03 20:54:36 +00:00
2024-11-06 11:53:33 +00:00
2024-11-06 11:53:33 +00:00
2024-12-03 20:54:36 +00:00
2024-12-03 20:54:36 +00:00
2024-12-03 20:54:36 +00:00
2024-12-03 20:54:36 +00:00
2025-01-26 14:43:51 +00:00
2025-02-14 11:03:33 -08:00
2025-01-26 14:43:51 +00:00
2024-11-06 11:53:33 +00:00
2025-03-03 07:17:01 +07:00
2025-03-12 14:46:00 -07:00
2024-11-06 11:53:33 +00:00
2025-01-13 09:28:27 -08:00
2025-03-17 11:04:38 -07:00
2025-04-04 11:14:49 -07:00
2025-04-16 11:44:55 -07:00
2025-02-27 08:41:46 -08:00
2024-11-06 11:53:33 +00:00
2025-02-21 07:15:02 -05:00
2025-02-14 11:03:33 -08:00
2025-02-04 19:09:57 -05:00
2025-04-16 00:08:59 +08:00
2025-04-16 00:08:59 +08:00
2024-10-31 13:50:02 -04:00
2025-06-17 08:14:05 -07:00
2025-02-21 07:15:02 -05:00
2025-01-25 11:50:14 -08:00
2024-11-06 11:53:33 +00:00
2025-04-15 08:08:45 +08:00
2025-04-22 11:30:29 -04:00
2025-05-17 11:06:37 -07:00
2024-10-31 13:50:02 -04:00
2025-01-21 10:33:03 -05:00
2025-02-21 07:15:02 -05:00
2024-11-06 11:53:33 +00:00
2025-04-16 00:08:59 +08:00
2024-11-18 14:09:31 -08:00
2025-03-12 08:18:51 -07:00
2025-04-04 11:14:49 -07:00
2024-11-06 11:53:33 +00:00
2025-02-21 07:15:02 -05:00
2025-01-17 09:48:03 -08:00
2025-02-14 11:03:33 -08:00
2025-02-14 11:03:33 -08:00
2024-11-06 11:53:33 +00:00
2025-04-10 08:30:43 -07:00
2024-11-06 11:53:33 +00:00
2025-01-18 11:48:53 -08:00
2024-10-14 07:00:56 -07:00
2025-01-18 10:23:20 +08:00
2025-02-04 19:09:57 -05:00
2025-02-04 19:09:57 -05:00
2024-11-22 16:10:17 -05:00
2025-04-15 13:57:53 -07:00
2024-09-21 15:41:06 -07:00
2024-09-21 15:41:06 -07:00
2025-01-29 10:23:09 +00:00
2024-11-06 11:53:33 +00:00
2025-03-12 08:18:51 -07:00
2025-05-15 10:08:05 +01:00
2025-04-11 09:42:34 -04:00
2025-02-14 11:03:33 -08:00
2025-05-29 10:09:49 -07:00
2025-05-18 14:26:36 -07:00
2025-05-18 12:59:36 -07:00
2025-05-17 11:06:37 -07:00
2025-03-12 08:18:51 -07:00
2025-03-12 14:46:00 -07:00
2025-04-22 11:30:29 -04:00
2025-04-16 00:08:59 +08:00
2025-02-21 07:15:02 -05:00
2024-12-20 10:39:45 +00:00
2024-09-27 04:14:28 -07:00
2025-02-21 07:15:02 -05:00
2024-12-05 14:36:40 +01:00
2024-11-06 11:53:33 +00:00
2025-04-22 11:30:29 -04:00
2025-04-11 09:42:34 -04:00
2025-01-20 14:22:20 -08:00
2025-02-14 11:03:33 -08:00
2024-12-03 20:54:36 +00:00
2025-05-18 12:59:36 -07:00
2025-03-12 08:18:51 -07:00
2024-11-25 07:58:19 -08:00
2024-09-27 12:54:56 -07:00
2024-11-06 11:53:33 +00:00
2025-01-21 10:33:03 -05:00
2025-03-25 04:35:33 -07:00
2025-03-14 07:04:55 -07:00
2025-04-16 00:08:59 +08:00
2024-12-18 19:06:34 +01:00
2024-12-05 14:36:40 +01:00
2024-12-03 20:54:36 +00:00
2024-12-03 20:54:36 +00:00
2025-02-21 07:15:02 -05:00
2024-11-06 11:53:33 +00:00
2024-10-31 13:50:02 -04:00
2024-12-03 20:54:36 +00:00
2025-05-01 14:28:20 -07:00
2025-04-24 11:08:17 -07:00
2025-02-14 11:03:33 -08:00
2025-05-18 12:59:36 -07:00
2025-02-21 07:15:02 -05:00
2025-03-12 14:46:00 -07:00
2025-05-14 08:18:23 -07:00
2025-03-12 08:18:51 -07:00
2025-02-04 19:09:57 -05:00
2025-02-04 19:09:57 -05:00
2024-11-06 11:53:33 +00:00
2024-11-06 11:53:33 +00:00
2024-12-03 20:54:36 +00:00
2024-11-06 11:53:33 +00:00
2024-10-31 13:23:43 -04:00
2024-11-06 11:53:33 +00:00
2024-11-06 11:53:33 +00:00
2024-11-06 11:53:33 +00:00
2024-11-06 11:53:33 +00:00
2024-11-06 11:53:33 +00:00
2025-04-04 11:14:49 -07:00
2025-04-04 11:14:49 -07:00
2025-04-04 11:14:49 -07:00
2024-11-06 11:53:33 +00:00
2024-11-06 11:53:33 +00:00
2024-11-06 11:53:33 +00:00
2025-04-16 00:08:59 +08:00
2025-04-11 09:42:34 -04:00
2025-05-17 11:06:37 -07:00
2025-02-14 11:03:33 -08:00
2024-11-06 11:53:33 +00:00
2024-11-06 11:53:33 +00:00
2025-02-14 11:03:33 -08:00
2025-03-31 19:33:40 -04:00
2025-03-24 09:29:18 -04:00
2025-02-26 14:17:11 -08:00
2024-11-06 11:53:33 +00:00
2024-11-06 11:53:33 +00:00
2024-11-06 11:53:33 +00:00
2024-10-24 07:47:23 -07:00
2025-04-04 11:14:49 -07:00
2024-12-03 20:54:36 +00:00
2025-05-12 20:03:45 +08:00
2024-12-03 20:54:36 +00:00
2024-10-21 13:37:44 +01:00
2024-12-03 20:54:36 +00:00
2025-03-12 08:18:51 -07:00
2025-04-04 11:14:49 -07:00
2025-04-04 11:14:49 -07:00
2025-04-22 12:11:01 -04:00
2025-03-14 14:11:09 -07:00
2025-05-20 07:31:16 -04:00
2025-04-11 09:42:34 -04:00
2024-12-03 20:54:36 +00:00
2025-01-28 05:45:13 -08:00
2024-12-03 20:54:36 +00:00
2025-02-21 07:15:02 -05:00
2024-11-06 11:53:33 +00:00
2025-04-02 19:04:07 +08:00
2025-03-25 18:58:54 +08:00
2025-04-08 22:29:11 +08:00
2025-04-10 18:54:45 +08:00
2025-03-20 21:04:30 +08:00
2025-04-10 23:07:45 +08:00
2025-03-20 11:31:49 +08:00
2025-04-15 23:03:55 +08:00
2025-04-11 03:01:09 -07:00
2025-02-14 11:03:33 -08:00
2024-10-21 09:16:16 -07:00
2025-02-03 06:05:19 -08:00
2025-03-18 10:01:13 -07:00
2025-03-12 08:18:51 -07:00
2025-02-05 08:54:09 +00:00
2024-11-06 11:53:33 +00:00
2025-04-01 14:30:06 -04:00
2025-04-04 11:14:49 -07:00
2024-11-06 11:53:33 +00:00
2025-04-11 09:42:34 -04:00
2025-02-21 07:15:02 -05:00
2025-04-16 00:08:59 +08:00
2024-11-06 11:53:33 +00:00
2025-04-04 11:14:49 -07:00
2024-11-06 11:53:33 +00:00
2025-01-17 09:48:03 -08:00
2025-02-21 07:15:02 -05:00
2024-10-14 07:00:56 -07:00
2024-10-23 10:46:38 -07:00
2025-04-10 12:06:38 -04:00
2025-04-04 11:14:49 -07:00
2025-04-04 11:14:49 -07:00
2025-04-15 13:17:46 -07:00
2025-04-10 12:06:38 -04:00
2025-04-15 13:17:46 -07:00
2025-05-17 11:06:37 -07:00
2025-03-28 12:45:53 -07:00
2025-03-12 08:18:51 -07:00
2025-04-16 06:23:44 -07:00
2024-11-06 11:53:33 +00:00
2024-11-06 11:53:33 +00:00
2024-12-04 16:36:00 +00:00
2024-11-06 11:53:33 +00:00
2025-02-21 07:15:02 -05:00
2024-11-06 11:53:33 +00:00
2025-04-11 11:40:41 -07:00
2025-02-28 11:50:50 -08:00
2025-02-14 11:03:33 -08:00
2024-11-06 11:53:33 +00:00
2025-02-28 13:17:14 -08:00
2025-05-18 12:59:36 -07:00
2024-12-03 20:54:36 +00:00
2025-01-17 09:48:03 -08:00
2025-04-16 00:08:59 +08:00
2024-10-21 12:25:39 -07:00
2025-04-01 14:30:06 -04:00
2025-03-12 08:18:51 -07:00
2025-06-17 08:14:05 -07:00
2025-03-12 08:18:51 -07:00
2025-04-16 00:08:59 +08:00
2024-10-21 12:25:39 -07:00
2025-04-11 09:42:34 -04:00
2024-11-06 11:53:33 +00:00
2025-01-17 09:48:03 -08:00
2025-04-02 07:26:40 -07:00