allenai

/

OLMo-2-0325-32B-Instruct-GGUF

like 6

Follow

Ai2 2.8k

https://github.com/ggml-org/llama.cpp/issues/12376

\n

converts and quantizes no problem, but fails to run.

\n

llama_model_load: error loading model: check_tensor_dims: tensor 'blk.0.attn_k_norm.weight' has wrong shape; expected 5120, got 1024, 1, 1, 1

\n","updatedAt":"2025-03-13T23:33:02.599Z","author":{"_id":"6435718aaaef013d1aec3b8b","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6435718aaaef013d1aec3b8b/XKf-8MA47tjVAM6SCX0MP.jpeg","fullname":"Bartowski","name":"bartowski","type":"user","isPro":true,"isHf":false,"isMod":false,"followerCount":5213}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8545901775360107},"editors":["bartowski"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/6435718aaaef013d1aec3b8b/XKf-8MA47tjVAM6SCX0MP.jpeg"],"reactions":[],"isReport":false}},{"id":"67d36baf2f1786233a2da2a6","author":{"_id":"639e6534f87da5e2eb1e6910","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/639e6534f87da5e2eb1e6910/yfE1kVUHF_whlulqo9QVm.jpeg","fullname":"Aman Rangapur","name":"amanrangapur","type":"user","isPro":false,"isHf":false,"isMod":false,"followerCount":3,"isOwner":false,"isOrgMember":true},"createdAt":"2025-03-13T23:35:11.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Hey @bartowski, Is this issue only for Q3's?","html":"

Hey \n\n@bartowski\n\t, Is this issue only for Q3's?

\n","updatedAt":"2025-03-13T23:35:11.778Z","author":{"_id":"639e6534f87da5e2eb1e6910","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/639e6534f87da5e2eb1e6910/yfE1kVUHF_whlulqo9QVm.jpeg","fullname":"Aman Rangapur","name":"amanrangapur","type":"user","isPro":false,"isHf":false,"isMod":false,"followerCount":3}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7659053802490234},"editors":["amanrangapur"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/639e6534f87da5e2eb1e6910/yfE1kVUHF_whlulqo9QVm.jpeg"],"reactions":[],"isReport":false}},{"id":"67d36bd640aba403b6401cae","author":{"_id":"6435718aaaef013d1aec3b8b","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6435718aaaef013d1aec3b8b/XKf-8MA47tjVAM6SCX0MP.jpeg","fullname":"Bartowski","name":"bartowski","type":"user","isPro":true,"isHf":false,"isMod":false,"followerCount":5213,"isOwner":false,"isOrgMember":false},"createdAt":"2025-03-13T23:35:50.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"no it's for all sizes sadly!\n\nBF16 also failed in the same way","html":"

no it's for all sizes sadly!

\n

BF16 also failed in the same way

\n","updatedAt":"2025-03-13T23:35:50.448Z","author":{"_id":"6435718aaaef013d1aec3b8b","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6435718aaaef013d1aec3b8b/XKf-8MA47tjVAM6SCX0MP.jpeg","fullname":"Bartowski","name":"bartowski","type":"user","isPro":true,"isHf":false,"isMod":false,"followerCount":5213}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9836929440498352},"editors":["bartowski"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/6435718aaaef013d1aec3b8b/XKf-8MA47tjVAM6SCX0MP.jpeg"],"reactions":[{"reaction":"😔","users":["amanrangapur","ubergarm"],"count":2}],"isReport":false}},{"id":"67d36c75c6c021f674bf2b39","author":{"_id":"6435718aaaef013d1aec3b8b","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6435718aaaef013d1aec3b8b/XKf-8MA47tjVAM6SCX0MP.jpeg","fullname":"Bartowski","name":"bartowski","type":"user","isPro":true,"isHf":false,"isMod":false,"followerCount":5213,"isOwner":false,"isOrgMember":false},"createdAt":"2025-03-13T23:38:29.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"I'll download Q8_0 to be extra sure, but I think it's safe to say it applies to all quants if it happens to BF16","html":"

I'll download Q8_0 to be extra sure, but I think it's safe to say it applies to all quants if it happens to BF16

\n","updatedAt":"2025-03-13T23:38:29.715Z","author":{"_id":"6435718aaaef013d1aec3b8b","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6435718aaaef013d1aec3b8b/XKf-8MA47tjVAM6SCX0MP.jpeg","fullname":"Bartowski","name":"bartowski","type":"user","isPro":true,"isHf":false,"isMod":false,"followerCount":5213}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9767382740974426},"editors":["bartowski"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/6435718aaaef013d1aec3b8b/XKf-8MA47tjVAM6SCX0MP.jpeg"],"reactions":[],"isReport":false}},{"id":"67d36fa16bd1de55f8714a4e","author":{"_id":"6435718aaaef013d1aec3b8b","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6435718aaaef013d1aec3b8b/XKf-8MA47tjVAM6SCX0MP.jpeg","fullname":"Bartowski","name":"bartowski","type":"user","isPro":true,"isHf":false,"isMod":false,"followerCount":5213,"isOwner":false,"isOrgMember":false},"createdAt":"2025-03-13T23:52:01.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Yup, Q8_0 breaks in the same way @amanrangapur ","html":"

Yup, Q8_0 breaks in the same way \n\n@amanrangapur\n\t

\n","updatedAt":"2025-03-13T23:52:01.618Z","author":{"_id":"6435718aaaef013d1aec3b8b","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6435718aaaef013d1aec3b8b/XKf-8MA47tjVAM6SCX0MP.jpeg","fullname":"Bartowski","name":"bartowski","type":"user","isPro":true,"isHf":false,"isMod":false,"followerCount":5213}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.5700713396072388},"editors":["bartowski"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/6435718aaaef013d1aec3b8b/XKf-8MA47tjVAM6SCX0MP.jpeg"],"reactions":[],"isReport":false}}],"pinned":false,"locked":false,"collection":"discussions","isPullRequest":false,"isReport":false},"primaryEmailConfirmed":false,"repo":{"name":"allenai/OLMo-2-0325-32B-Instruct-GGUF","type":"model"},"discussionRole":0,"acceptLanguages":["en","*"],"disableDiscussionClosingAndCommentHiding":false,"hideComments":true}">