Device Agnostic Pipeline #140
Conversation
There was a problem hiding this comment.
What's the purpose of this PR?
Have an example a user can try out on any HW and get decent performance?
We don't expect anyone to really use this for anything right?
The GEMM isn't really device agonistic, is it? It's more that likely any GPU a user would run on has features assumed by the SM_70 pipeline.
If so would it be sufficient to only add the example, but not the new builder and instead directly use the SM_70 builder as a baseline for most current GPUs?
| /// Prints the usage statement. | ||
| std::ostream & print_usage(std::ostream &out) const { | ||
|
|
||
| out << "PVC GEMM Example\n\n" |
| /// Prints the usage statement. | ||
| std::ostream & print_usage(std::ostream &out) const { | ||
|
|
||
| out << "PVC GEMM Example\n\n" |
| // Run examples | ||
| // | ||
|
|
||
| // The KernelHardwareInfo struct holds the number of EUs on the GPU with a given device ID. This |
There was a problem hiding this comment.
"EU" isn't a general term
| #endif | ||
|
|
||
| using TiledMMA = TiledMMA<MMA_Atom<UniversalFMA<ElementAccumulator, ElementA, ElementB, ElementAccumulator>>, | ||
| Layout<Shape<_4, _4, _1>>>; |
There was a problem hiding this comment.
This would result in a work-group size of 16, it's small enough that it would run on any device, hence the size. No other reason in particular
The SM70 mainloop is device agnostic, it implements a tiled GEMM algorithm, with data being blocked in shared memory and registers. With us passing the UniversalCopy and UniversalMMA, this would become a truly device agnostic gemm.
SM70 does not have a collective builder. Also, I believe the idea is that the API accepts something like a |
Adds
CollectiveMMAand aCollectiveBuilderAPI for device agnostic pipeline.This piggybacks off of the SM_70 2 stage gemm pipeline, with blocking in SMem and RMem, to get somewhat performant
gemmon any device.