https://jwhogg.github.io/tags/research/Recent content in Research on jwhoggHugoen-usTue, 18 Jun 2024 00:00:00 +0000https://jwhogg.github.io/articles/ml_inference_browser/Tue, 18 Jun 2024 00:00:00 +0000https://jwhogg.github.io/articles/ml_inference_browser/<p>Recently, I stumbled across a <a href="https://whisper.ggerganov.com/">guy who ported</a> OpenAI&rsquo;s <a href="https://github.com/openai/whisper">Whisper</a> model into c++, in various sizes, allowing the model to be run on-device, at impressive speed.</p> <!-- <span class="text-gray-500 text-sm"> [(source)](https://github.com/ggerganov/whisper.cpp) </span> --> <!--  --> <p>I went down a rabbit-hole, and found a whole family of popular models that had been ported to work on-device, from the browser:</p> <ul> <li><a href="https://websd.mlc.ai/#text-to-image-generation-demo">Stable difusion running in browser</a></li> <li><a href="https://github.com/jobergum/browser-ml-inference?tab=readme-ov-file">Text emotion prediction in browser</a></li> <li><a href="https://github.com/ggerganov/llama.cpp">Llama c++</a></li> <li><a href="https://webllm.mlc.ai/">Web LLM</a></li> <li><a href="https://hyuto.github.io/yolov5-onnxruntime-web/">YOLO in the browser</a></li> </ul> <p><figure> <img src="https://jwhogg.github.io/images/yolo_browser.webp#smaller" alt="yolo in the browser" loading="lazy" /> <figcaption>yolo in the browser</figcaption> </figure> </p>