huggingface/text-generation-inference
5
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2025-04-20 14:22:08 +00:00
Code Issues Packages Projects Releases Wiki Activity
git_v3.1.0
text-generation-inference/server/text_generation_server/layers/moe
History
Nicolas Patry cb747b33da
Add deepseekv3 (#2968) 
* Add fp8 support moe models

add deepseekv3

format codfe'

update dockerfile

update doc

* Small modifications.

* Moe kernels 0.8.1

* Upgrade to 0.8.1

* Fixing moe import.

* Black.

* Apply suggestions from code review

Co-authored-by: Mohit Sharma <mohit21sharma.ms@gmail.com>

* Fixing Mixtral + Nits.

* Put link to ref.

* Fix other call locations.

* Scoring func `softmax` is the only one that works.

---------

Co-authored-by: Mohit Sharma <mohit21sharma.ms@gmail.com>
2025-01-30 16:40:25 +01:00
..
__init__.py Add deepseekv3 (#2968) 2025-01-30 16:40:25 +01:00
fp8.py Add deepseekv3 (#2968) 2025-01-30 16:40:25 +01:00
fused_moe_ipex.py fix moe in quantization path (#2935) 2025-01-22 14:36:15 +01:00
gptq_marlin.py Add deepseekv3 (#2968) 2025-01-30 16:40:25 +01:00
unquantized.py Add deepseekv3 (#2968) 2025-01-30 16:40:25 +01:00