Daniël de Kok 748764efb4 Add Phi-3 medium support (#2039) ... Add support for Phi-3-medium The main difference between the medium and mini models is that medium uses grouped query attention with a packed QKV matrix. This change adds support for GQA with packed matrixes to `Weights.get_weights_col_packed` and uses it for Phi-3. This also allows us to remove the custom implementation of GQA from dbrx attention loading.		2024-09-24 03:42:29 +00:00
..
attention	ROCm and sliding windows fixes (#2033)	2024-09-24 03:42:29 +00:00
awq	Refactor layers. (#1866)	2024-07-17 05:36:58 +00:00
gptq	Fix GPTQWeight import (#2020)	2024-09-24 03:34:15 +00:00
__init__.py	MLPSpeculator. (#1865)	2024-07-17 05:36:58 +00:00
bnb.py	Update torch import reference in bnb quantization (#1902)	2024-07-17 05:36:58 +00:00
conv.py	Refactor layers. (#1866)	2024-07-17 05:36:58 +00:00
eetq.py	Refactor layers. (#1866)	2024-07-17 05:36:58 +00:00
exl2.py	Add support for exl2 quantization	2024-09-24 03:19:39 +00:00
fp8.py	Refactor layers. (#1866)	2024-07-17 05:36:58 +00:00
layernorm.py	MI300 compatibility (#1764)	2024-07-17 05:36:58 +00:00
linear.py	Add support for Marlin-quantized models	2024-09-24 03:38:05 +00:00
marlin.py	Add support for Marlin-quantized models	2024-09-24 03:38:05 +00:00
medusa.py	fix: use path inside of speculator config (#1935)	2024-07-17 05:36:58 +00:00
mlp.py	MLPSpeculator. (#1865)	2024-07-17 05:36:58 +00:00
rotary.py	reenable xpu for tgi (#1939)	2024-07-17 05:36:58 +00:00
speculative.py	MLPSpeculator. (#1865)	2024-07-17 05:36:58 +00:00
tensor_parallel.py	Add Phi-3 medium support (#2039)	2024-09-24 03:42:29 +00:00