GitHub / AI-Hypercomputer/JetStream / commits
JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs welcome).
| SHA | Message | Author | Date | Stats |
|---|---|---|---|---|
| acd4f5ad | Update README with archival notice for Jetstream (#278) |
Vipan Nalla <3****a@u****m>
Committed by: GitHub <n****y@g****m> |
7 months ago | |
| 29329e8e | merge Merge pull request #275 from AI-Hypercomputer:log_prob_pytree_node | jetstream authors <n****y@g****m> | about 1 year ago | |
| daedc21c | log prob should be a pytree node | wenxindongwork <w****g@g****m> | about 1 year ago | |
| 261f2500 | merge Merge pull request #222 from AI-Hypercomputer:amangu-lora-3 | jetstream authors <n****y@g****m> | about 1 year ago | |
| 8839d1a5 | - JetStream changes for Jax based implementation of unified_lora_params for d... | Aman Gupta <a****0@g****m> | about 1 year ago | |
| 89acc8c4 | merge Merge pull request #271 from AI-Hypercomputer:lihao/fix | jetstream authors <n****y@g****m> | about 1 year ago | |
| 083f2c97 | Fix nltk errors | Lihao Ran <i****n@g****m> | over 1 year ago | |
| 2756c6fe | merge Merge pull request #269 from AI-Hypercomputer:yuyan-prefix-cache | jetstream authors <n****y@g****m> | over 1 year ago | |
| 4aafd767 | merge Merge pull request #268 from AI-Hypercomputer:yuyan-prefix-cache-benchmark | jetstream authors <n****y@g****m> | over 1 year ago | |
| f40d0da8 | Refactor(PrefixCache): New load API, per-layer Tries, async ops & stats | Yuyan Peng <y****g@g****m> | over 1 year ago | |
| bbfb5bd2 | feat: Add prefix cache benchmark | Yuyan Peng <y****g@g****m> | over 1 year ago | |
| 219e5a17 | merge Merge pull request #266 from AI-Hypercomputer:lihao/bos | jetstream authors <n****y@g****m> | over 1 year ago | |
| 97a30110 | merge Merge pull request #267 from AI-Hypercomputer:gsutil-bug-fix | jetstream authors <n****y@g****m> | over 1 year ago | |
| 0ba366dd | fix gsutil | Vijaya <m****h@g****m> | over 1 year ago | |
| 508a9477 | Allow users to specify whether or not to add bos token | Lihao Ran <i****n@g****m> | over 1 year ago | |
| 47ed5a98 | merge Merge pull request #265 from AI-Hypercomputer:vij-upload-manifest | jetstream authors <n****y@g****m> | over 1 year ago | |
| ae77b834 | upload manifest to gcs and compare with golden numbers | Vijaya <m****h@g****m> | over 1 year ago | |
| 63c0d3eb | merge Merge pull request #264 from AI-Hypercomputer:yuyan-upload-manifest | jetstream authors <n****y@g****m> | over 1 year ago | |
| 424d52d3 | send manifest file and llama test | Vijaya <m****h@g****m> | over 1 year ago | |
| 58abceb4 | merge Merge pull request #263 from AI-Hypercomputer:llama_runs_benchmarks | jetstream authors <n****y@g****m> | over 1 year ago | |
| 65a7c91b | llama benchmarks | Vijaya <m****h@g****m> | over 1 year ago | |
| 8fe9a20b | merge Merge pull request #262 from AI-Hypercomputer:yuyan-fix-attachment | jetstream authors <n****y@g****m> | over 1 year ago | |
| b9d8a89e | Fix stable stack benchmark report path | Yuyan Peng <y****g@g****m> | over 1 year ago | |
| 08454d3c | merge Merge pull request #261 from AI-Hypercomputer:karan-model_ckpt_conversion-fix | jetstream authors <n****y@g****m> | over 1 year ago | |
| ac1d9b03 | model_ckpt_conversion: use python module, enable gcloud checkpoint downloading | Karan Goel <k****l@g****m> | over 1 year ago | |
| 1e1e9d46 | merge Merge pull request #256 from AI-Hypercomputer:yuyan-fix-test-hang | jetstream authors <n****y@g****m> | over 1 year ago | |
| c5e54e12 | merge Merge pull request #260 from AI-Hypercomputer:maxtext_logp | jetstream authors <n****y@g****m> | over 1 year ago | |
| 7cec37fc | return logp | wenxindongwork <w****g@g****m> | over 1 year ago | |
| 77735f1f | Fix stable stack test hang while server crash and gcloud auth |
Yuyan Peng <y****g@g****m>
Committed by: Vijaya <m****h@g****m> |
over 1 year ago | |
| 5c398f64 | merge Merge pull request #255 from AI-Hypercomputer:yuyan-stable-stack2 | jetstream authors <n****y@g****m> | over 1 year ago | |
| f7427db8 | merge Merge pull request #243 from AI-Hypercomputer:yuyan-prefix-cache | jetstream authors <n****y@g****m> | over 1 year ago | |
| f71349ac | merge Merge pull request #254 from AI-Hypercomputer:yuyan-stable-stack | jetstream authors <n****y@g****m> | over 1 year ago | |
| eb2e377e | ci: Add Jetsteram Maxtext stable stack workflow | Yuyan Peng <y****g@g****m> | over 1 year ago | |
| a789475a | feat(jetstream-maxtext): Add stable stack Docker image and build scripts | Yuyan Peng <y****g@g****m> | over 1 year ago | |
| 9cb7785d | Refactor: Execute Maxtext inference as a module via `python3 -m` |
Yijia Jin <y****j@g****m>
Committed by: jetstream authors <n****y@g****m> |
over 1 year ago | |
| 51ab94c2 | merge Merge pull request #253 from AI-Hypercomputer:fix_moe_runner | jetstream authors <n****y@g****m> | over 1 year ago | |
| a84be2d2 | merge Merge pull request #251 from AI-Hypercomputer:vipannalla-patch-1 | jetstream authors <n****y@g****m> | over 1 year ago | |
| 3487fdf1 | minor fixes | Vijaya <m****h@g****m> | over 1 year ago | |
| cb86ae26 | Refactor: Execute Maxtext inference as a module via `python3 -m` |
Yijia Jin <y****j@g****m>
Committed by: jetstream authors <n****y@g****m> |
over 1 year ago | |
| fcc164bb | Update CODEOWNERS |
Vipan Nalla <3****a@u****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| c856af56 | merge Merge pull request #250 from AI-Hypercomputer:vij_moe_runner | jetstream authors <n****y@g****m> | over 1 year ago | |
| 40585107 | update_moe_tests | Vijaya <m****h@g****m> | over 1 year ago | |
| 7ca7c584 | merge Merge pull request #249 from AI-Hypercomputer:amangu-lora | jetstream authors <n****y@g****m> | over 1 year ago | |
| bf3483f4 | 1) Adding __init__.py files for newly added lora and lora/test folders. | aman2930 <a****0@g****m> | over 1 year ago | |
| 50205851 | merge Merge pull request #248 from AI-Hypercomputer:yijiaj/synctest | jetstream authors <n****y@g****m> | over 1 year ago | |
| 7a6e9ad8 | nit | Yijia J <y****j@g****m> | over 1 year ago | |
| da199e34 | Syncing RevId |
jetstream authors <n****y@g****m>
Committed by: Yijia J <y****j@g****m> |
over 1 year ago | |
| f8ec421f | Add Github Runner and tests (#246) |
Vijaya Singh <m****h@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| 035e13ab | Test if copybara revId can be changed |
Yijia Jin <y****j@g****m>
Committed by: jetstream authors <n****y@g****m> |
over 1 year ago | |
| 22b37d3f | merge Merge pull request #247 from AI-Hypercomputer:yijiaj/test2 | jetstream authors <n****y@g****m> | over 1 year ago | |
| f213ffe8 | test | Yijia J <y****j@g****m> | over 1 year ago | |
| d028b239 | COPYBARA_INTEGRATE_REVIEW=https://github.com/AI-Hypercomputer/JetStream/pull/... |
Yijia <j****4@g****m>
Committed by: Yijia J <y****j@g****m> |
over 1 year ago | |
| 34aa3713 | Reverts changelist 743619652 |
jetstream authors <n****y@g****m>
Committed by: Yijia J <y****j@g****m> |
over 1 year ago | |
| ad7e4947 | feat(core, engine): Implement prefix caching for chunked prefill | Yuyan Peng <y****g@g****m> | over 1 year ago | |
| 082c0ac5 | Supporting Multi-LoRA inferencing via JetStream server (#221) |
Aman Gupta <4****0@u****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| 045c9a1e | Adding PyTests in JetStream unit test workflow for code coverage. (#242) |
Aman Gupta <4****0@u****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| f4a13005 | Retry grpc async request (#240) |
yuyanpeng-google <y****g@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| ebcab3af | Move prefix cache from MaxText (#239) |
yuyanpeng-google <y****g@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| a2b33332 | Updating Makefile to include pytests in coverage. (#238) |
Aman Gupta <4****0@u****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| 11341e22 | Always print server starup messages (#237) |
Vipan Nalla <3****a@u****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| 8aa6a9ec | Added explicit pathwaysutils.initialize() call to JetStream (#233) |
Yu Li <w****u@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| 838f6148 | Modify MMLU prompt template to improve accuracy on DeepSeek models; Eā¦ (#235) |
bzgoogle <b****z@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| 0aa437f4 |
Add yml file of adding pull_ready tags for copybara flow. The file is almost ...
Co-authored-by: Yijia J <y****j@g****m> |
Yijia <j****4@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| e5390076 | Refactor chunked prefill (#232) |
yuyanpeng-google <y****g@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| b8b9cb2e | Fix chunked prefill regression (#231) |
yuyanpeng-google <y****g@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| 351462ec | Support long context dataset accuracy measurement (#230) |
Lumosis <3****s@u****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| 9d19631b | Added new HuggingFaceTokenizer to token_utils and updated TokenizerParameters... |
gpolovets1 <g****s@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| b8ad7272 | [Performance] Not set request uuid to resolve performance regression (#228) |
wyzhang <w****g@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| d95a4b74 | Add long context dataset benchmark support (#227) |
Lumosis <3****s@u****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| 5240b3b4 | Add post-processing functions to extract math500 answers (#225) |
bzgoogle <b****z@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| f5c19bc1 | fixes to formatting and benchmark eval (#224) |
vivianrwu <v****u@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| a75f64fe | Adding a parameter=lora_input_adapters_path, to start the server with path if... |
Aman Gupta <4****0@u****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| 982569f8 | Add the llama2-70b model (#217) |
Yong Xia <y****x@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| 2c4f6d9c | Add unit tests to increase coverage for multi-sampling (#220) |
Lumosis <3****s@u****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| 698f33fc | Chunked Prefill (#188) |
Vijaya Singh <m****h@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| f602565a | Kill the generate thread when getting a "None" new request (#218) |
Lumosis <3****s@u****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| 1d6b4563 |
Revert accidental change - back to #216
Co-authored-by: Yijia J <y****j@g****m> |
Yijia <j****4@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| 00dc5a61 | merge Merge pull request #183 from AI-Hypercomputer:yijiaj/test | jetstream authors <n****y@g****m> | over 1 year ago | |
| 951b3ef8 | Require at least 1GB of total HBM from all devices (#216) |
Yong Xia <y****x@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| 80bfefcc | Add multi-sampling functionality (#215) |
Lumosis <3****s@u****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| 84a67d14 | Update setup.py with latest release tag |
Vipan Nalla <3****a@u****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| 5a74bcad | Add request id into active request (#209) |
RupengLiu <r****u@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| 66ee11a3 | [refactor] clean up model executor (#214) |
Yong Xia <y****x@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| 9729700f | [cleanup] Further clean up on prefill/inference/postprocess functions (#213) |
Yong Xia <y****x@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| 297a807a | [style] Use single underscore prefix for private data/functions (#212) |
Yong Xia <y****x@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| 76b2d908 | [cleanup] Refactor kv cache storage/manager (#211) |
Yong Xia <y****x@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| 5dd2b91a | [cleanup] Refactor prefill functions (#210) |
Yong Xia <y****x@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| 345bcaf6 | [cleanup] Clean up logs (#208) |
Yong Xia <y****x@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| cddb3672 | 1> Add mmlu dataset; 2> fix minor bug for mmlu prompt; 3> add scikit-ā¦ (#203) |
bzgoogle <b****z@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| 577e0e21 | Fix the Math benchmark issues. (#207) |
Lance Wang <l****g@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| 4f503322 | [cleanup] Move all inference config parameters into one file (#204) |
Yong Xia <y****x@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| 947a947c | Fix the benchmark_serving minor error. (#202) |
Lance Wang <l****g@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| 85bcc3ad | [test] Move unit tests to the same folder as their source code (#200) |
Yong Xia <y****x@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| ec77720c | [cleanup/fix] Make inference parameters explicit, etc. (#198) |
Yong Xia <y****x@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| 85064bd1 | [Deepseek] enable mmlu benchmark (#193) |
bzgoogle <b****z@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| 01fcc2a0 | Add Math-500 dataset from Huggingface. (#199) |
Lance Wang <l****g@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| 8cae8a17 | [benchmark] Increase offline benchmarking batch size from 160 to 320 (#196) |
Yong Xia <y****x@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| 7f831bc7 | [test] Add a unit test for paged attention kernel (#195) |
Yong Xia <y****x@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| 88a2af46 | [test] fix unit test breaks (#194) |
Yong Xia <y****x@g****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago | |
| 2527fe42 |
Add comprehensive debugging messages. (#182)
Co-authored-by: Lihao Ran <r****o@g****m> |
Lumosis <3****s@u****m>
Committed by: GitHub <n****y@g****m> |
over 1 year ago |