Hugging Face's logo

microsoft
/
Phi-4-multimodal-instruct

Automatic Speech Recognition
Transformers
Safetensors
phi4mm
text-generation
nlp
code
audio
speech-summarization
speech-translation
visual-question-answering
phi-4-multimodal
phi
phi-4-mini
custom_code
Model card Files Community
44
https://huggingface.co/microsoft/Phi-4-multimodal-instruct-onnx

\n","updatedAt":"2025-03-01T12:42:23.771Z","author":{"_id":"640f1f4f06c3b5ca883f3900","avatarUrl":"/avatars/24a8c63c897efdd980ef9d4805cbff7b.svg","fullname":"Lim Chee Kin","name":"limcheekin","type":"user","isPro":false,"isHf":false,"isMod":false,"followerCount":28}},"numEdits":1,"identifiedLanguage":{"language":"en","probability":0.9827557802200317},"editors":["limcheekin"],"editorAvatarUrls":["/avatars/24a8c63c897efdd980ef9d4805cbff7b.svg"],"reactions":[],"isReport":false}},{"id":"67c5d8aa04933b046c9a07bf","author":{"_id":"5f3ec133a4dd343b63a632dd","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1597948137099-noauth.jpeg","fullname":"Nguyen Bach","name":"nguyenbh","type":"user","isPro":false,"isHf":false,"isMod":false,"followerCount":31,"isOwner":false,"isOrgMember":true},"createdAt":"2025-03-03T16:28:26.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"@limcheekin You may find this discussion helpful\nhttps://huggingface.co/microsoft/Phi-4-multimodal-instruct/discussions/7#67c4d764491ec4e926ed9d84","html":"

\n\n@limcheekin\n\t You may find this discussion helpful
https://huggingface.co/microsoft/Phi-4-multimodal-instruct/discussions/7#67c4d764491ec4e926ed9d84

\n","updatedAt":"2025-03-03T16:28:26.869Z","author":{"_id":"5f3ec133a4dd343b63a632dd","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1597948137099-noauth.jpeg","fullname":"Nguyen Bach","name":"nguyenbh","type":"user","isPro":false,"isHf":false,"isMod":false,"followerCount":31}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.44218549132347107},"editors":["nguyenbh"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1597948137099-noauth.jpeg"],"reactions":[],"isReport":false}},{"id":"67c9e877d956d50870f02ebe","author":{"_id":"668ae510c1c9eaffe41f1069","avatarUrl":"/avatars/cfdd3abd0cce00aedd572c7b82ef57ee.svg","fullname":"Non","name":"Bash82","type":"user","isPro":false,"isHf":false,"isMod":false,"isOwner":false,"isOrgMember":false},"createdAt":"2025-03-06T18:24:55.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Is speaker diarization possible?","html":"

Is speaker diarization possible?

\n","updatedAt":"2025-03-06T18:24:55.120Z","author":{"_id":"668ae510c1c9eaffe41f1069","avatarUrl":"/avatars/cfdd3abd0cce00aedd572c7b82ef57ee.svg","fullname":"Non","name":"Bash82","type":"user","isPro":false,"isHf":false,"isMod":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8537078499794006},"editors":["Bash82"],"editorAvatarUrls":["/avatars/cfdd3abd0cce00aedd572c7b82ef57ee.svg"],"reactions":[],"isReport":false}},{"id":"67d37d9441636d7ec43c8ce2","author":{"_id":"5f3ec133a4dd343b63a632dd","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1597948137099-noauth.jpeg","fullname":"Nguyen Bach","name":"nguyenbh","type":"user","isPro":false,"isHf":false,"isMod":false,"followerCount":31,"isOwner":false,"isOrgMember":true},"createdAt":"2025-03-14T00:51:32.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"@Bash82 the Phi-4-multimodal does not support speaker diarization. ","html":"

\n\n@Bash82\n\t the Phi-4-multimodal does not support speaker diarization.

\n","updatedAt":"2025-03-14T00:51:32.465Z","author":{"_id":"5f3ec133a4dd343b63a632dd","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1597948137099-noauth.jpeg","fullname":"Nguyen Bach","name":"nguyenbh","type":"user","isPro":false,"isHf":false,"isMod":false,"followerCount":31}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7568219900131226},"editors":["nguyenbh"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1597948137099-noauth.jpeg"],"reactions":[],"isReport":false}}],"pinned":false,"locked":false,"collection":"discussions","isPullRequest":false,"isReport":false},"repo":{"name":"microsoft/Phi-4-multimodal-instruct","type":"model"},"activeTab":"discussion","discussionRole":0}">

How to use it with LM Studio?

#3
by neokiller62 - opened
Discussion
https://huggingface.co/microsoft/Phi-4-multimodal-instruct-onnx

\n","updatedAt":"2025-03-01T12:42:23.771Z","author":{"_id":"640f1f4f06c3b5ca883f3900","avatarUrl":"/avatars/24a8c63c897efdd980ef9d4805cbff7b.svg","fullname":"Lim Chee Kin","name":"limcheekin","type":"user","isPro":false,"isHf":false,"isMod":false,"followerCount":28}},"numEdits":1,"identifiedLanguage":{"language":"en","probability":0.9827557802200317},"editors":["limcheekin"],"editorAvatarUrls":["/avatars/24a8c63c897efdd980ef9d4805cbff7b.svg"],"reactions":[],"isReport":false}},{"id":"67c5d8aa04933b046c9a07bf","author":{"_id":"5f3ec133a4dd343b63a632dd","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1597948137099-noauth.jpeg","fullname":"Nguyen Bach","name":"nguyenbh","type":"user","isPro":false,"isHf":false,"isMod":false,"followerCount":31,"isOwner":false,"isOrgMember":true},"createdAt":"2025-03-03T16:28:26.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"@limcheekin You may find this discussion helpful\nhttps://huggingface.co/microsoft/Phi-4-multimodal-instruct/discussions/7#67c4d764491ec4e926ed9d84","html":"

\n\n@limcheekin\n\t You may find this discussion helpful
https://huggingface.co/microsoft/Phi-4-multimodal-instruct/discussions/7#67c4d764491ec4e926ed9d84

\n","updatedAt":"2025-03-03T16:28:26.869Z","author":{"_id":"5f3ec133a4dd343b63a632dd","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1597948137099-noauth.jpeg","fullname":"Nguyen Bach","name":"nguyenbh","type":"user","isPro":false,"isHf":false,"isMod":false,"followerCount":31}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.44218549132347107},"editors":["nguyenbh"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1597948137099-noauth.jpeg"],"reactions":[],"isReport":false}},{"id":"67c9e877d956d50870f02ebe","author":{"_id":"668ae510c1c9eaffe41f1069","avatarUrl":"/avatars/cfdd3abd0cce00aedd572c7b82ef57ee.svg","fullname":"Non","name":"Bash82","type":"user","isPro":false,"isHf":false,"isMod":false,"isOwner":false,"isOrgMember":false},"createdAt":"2025-03-06T18:24:55.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Is speaker diarization possible?","html":"

Is speaker diarization possible?

\n","updatedAt":"2025-03-06T18:24:55.120Z","author":{"_id":"668ae510c1c9eaffe41f1069","avatarUrl":"/avatars/cfdd3abd0cce00aedd572c7b82ef57ee.svg","fullname":"Non","name":"Bash82","type":"user","isPro":false,"isHf":false,"isMod":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8537078499794006},"editors":["Bash82"],"editorAvatarUrls":["/avatars/cfdd3abd0cce00aedd572c7b82ef57ee.svg"],"reactions":[],"isReport":false}},{"id":"67d37d9441636d7ec43c8ce2","author":{"_id":"5f3ec133a4dd343b63a632dd","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1597948137099-noauth.jpeg","fullname":"Nguyen Bach","name":"nguyenbh","type":"user","isPro":false,"isHf":false,"isMod":false,"followerCount":31,"isOwner":false,"isOrgMember":true},"createdAt":"2025-03-14T00:51:32.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"@Bash82 the Phi-4-multimodal does not support speaker diarization. ","html":"

\n\n@Bash82\n\t the Phi-4-multimodal does not support speaker diarization.

\n","updatedAt":"2025-03-14T00:51:32.465Z","author":{"_id":"5f3ec133a4dd343b63a632dd","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1597948137099-noauth.jpeg","fullname":"Nguyen Bach","name":"nguyenbh","type":"user","isPro":false,"isHf":false,"isMod":false,"followerCount":31}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7568219900131226},"editors":["nguyenbh"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1597948137099-noauth.jpeg"],"reactions":[],"isReport":false}}],"pinned":false,"locked":false,"collection":"discussions","isPullRequest":false,"isReport":false},"primaryEmailConfirmed":false,"repo":{"name":"microsoft/Phi-4-multimodal-instruct","type":"model"},"discussionRole":0,"acceptLanguages":["en","*"],"disableDiscussionClosingAndCommentHiding":false,"hideComments":true}">
neokiller62

Can't find it in LM Studio download

asif00

Someone has to convert this to GGUF, it shouldn't take long.

supercharge19

After having a gguf version will it be able to do audio tasks, like ASR, speaker diarization, speech synthesis, as well?

limcheekin
•
edited 13 days ago

Someone has to convert this to GGUF, it shouldn't take long.

I guess GGUF not supported, otherwise we should see it by now.

Perhaps we can count on the ONNX version for CPU only environment for now.
Not sure LM Studio support it though.
https://huggingface.co/microsoft/Phi-4-multimodal-instruct-onnx

nguyenbh
 Microsoft org

@limcheekin You may find this discussion helpful
https://huggingface.co/microsoft/Phi-4-multimodal-instruct/discussions/7#67c4d764491ec4e926ed9d84

Bash82

Is speaker diarization possible?

nguyenbh
 Microsoft org

@Bash82 the Phi-4-multimodal does not support speaker diarization.

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment