GitHub / CouncilDataProject/cdp-backend / commits
Data storage utilities and processing pipelines used by CDP instances.
| SHA | Message | Author | Date | Stats |
|---|---|---|---|---|
| fda26250 |
ci(dependabot): bump extractions/setup-just from 1 to 2 (#243)
Co-authored-by: dependabot[bot] <4****]@u****m> Signed-off-by: dependabot[bot] <s****t@g****m> |
dependabot[bot] <4****]@u****m>
Committed by: GitHub <n****y@g****m> |
over 2 years ago | |
| db5858c8 |
ci(dependabot): bump actions/setup-python from 4 to 5 (#241)
Co-authored-by: dependabot[bot] <4****]@u****m> Signed-off-by: dependabot[bot] <s****t@g****m> |
dependabot[bot] <4****]@u****m>
Committed by: GitHub <n****y@g****m> |
almost 3 years ago | |
| 745ebca0 | Lint and format | Eva Maxfield Brown <e****n@g****m> | almost 3 years ago | |
| 96e3065a | Reduce tests rather than flaky | Eva Maxfield Brown <e****n@g****m> | almost 3 years ago | |
| c1c84697 | Introduce flaky to cover some tests | Eva Maxfield Brown <e****n@g****m> | almost 3 years ago | |
| 623871b9 | Try using non-transformer spacy model | Eva Maxfield Brown <e****n@g****m> | almost 3 years ago | |
| 06f618a8 | Try with spacy_transformers explicitly imported | Eva Maxfield Brown <e****n@g****m> | almost 3 years ago | |
| 8f72ae75 | Fix firebase setup during setup and deploy steps | Eva Maxfield Brown <e****n@g****m> | almost 3 years ago | |
| 99c5e3a5 | Update deps to fix whisper transcription | Eva Maxfield Brown <e****n@g****m> | almost 3 years ago | |
| d2f025b9 |
ci(dependabot): bump actions/checkout from 3 to 4 (#238)
Co-authored-by: dependabot[bot] <4****]@u****m> Signed-off-by: dependabot[bot] <s****t@g****m> |
dependabot[bot] <4****]@u****m>
Committed by: GitHub <n****y@g****m> |
about 3 years ago | |
| d6d71d17 | Fix file utils imports |
Eva Maxfield Brown <e****n@g****m>
Committed by: GitHub <n****y@g****m> |
over 3 years ago | |
| 69ff7754 | Use new __token__ for pypi | Eva Maxfield Brown <e****n@g****m> | over 3 years ago | |
| fcdf872f | Lint and format | Eva Maxfield Brown <e****n@g****m> | over 3 years ago | |
| 58a9fd76 | Handle error in event index pipeline when a term is blank (#237) |
Patrick Conant <p****k@p****m>
Committed by: GitHub <n****y@g****m> |
over 3 years ago | |
| a603be1f | feature/matter-text-extraction (#231) |
Sarah <3****7@u****m>
Committed by: GitHub <n****y@g****m> |
over 3 years ago | |
| 703d7f15 | Use requests stream and shutil.copyfileobj to constrain memory usage during r... |
Wes Hargrove <1****e@u****m>
Committed by: GitHub <n****y@g****m> |
over 3 years ago | |
| 09828303 | Remove Unneccessary Re-Encode If Video Is Already H264 (#234) |
Wes Hargrove <1****e@u****m>
Committed by: GitHub <n****y@g****m> |
over 3 years ago | |
| 6c1c1ff1 | feature/reduce-event-gather-complexity (#232) |
Eva Maxfield Brown <e****n@g****m>
Committed by: GitHub <n****y@g****m> |
over 3 years ago | |
| ad09d9c5 | Ignore lint error and fix session video uri call |
Eva Maxfield Brown <e****n@g****m>
Committed by: GitHub <n****y@g****m> |
over 3 years ago | |
| f049747a | Try to fix URLs missing www |
Eva Maxfield Brown <e****n@g****m>
Committed by: GitHub <n****y@g****m> |
over 3 years ago | |
| 0b860f58 | Try with no strict JSON load |
Eva Maxfield Brown <e****n@g****m>
Committed by: GitHub <n****y@g****m> |
over 3 years ago | |
| 4743ab63 | Add back accidentily removed imports |
Eva Maxfield Brown <e****n@g****m>
Committed by: GitHub <n****y@g****m> |
over 3 years ago | |
| 9f270545 | New faster-whisper from pypi (with word timestamps) |
Eva Maxfield Brown <e****n@g****m>
Committed by: GitHub <n****y@g****m> |
over 3 years ago | |
| 200d0e73 | docs/improve-dev-infra-setup (#229) |
Isaac Na <i****9@g****m>
Committed by: GitHub <n****y@g****m> |
over 3 years ago | |
| f63a3436 | Pin openai-whisper | Eva Maxfield Brown <e****n@g****m> | over 3 years ago | |
| d1fb41e6 | Do not allow breaking fireo change | Eva Maxfield Brown <e****n@g****m> | over 3 years ago | |
| 8e54f4ab | Move backoff dep to primary deps | Eva Maxfield Brown <e****n@g****m> | over 3 years ago | |
| e63c25db | Also catch comma doc sentences | Eva Maxfield Brown <e****n@g****m> | over 3 years ago | |
| 7727bcc2 | Whisper: fix mult spaces & period, no double clean | Eva Maxfield Brown <e****n@g****m> | over 3 years ago | |
| 3566dea5 | Add backoff to db validators | Eva Maxfield Brown <e****n@g****m> | over 3 years ago | |
| 576fda43 | Add time delay between each youtube test | Eva Maxfield Brown <e****n@g****m> | over 3 years ago | |
| d811ae15 | Use normal python function and backoff lib instead of prefect task for get_ev... | Eva Maxfield Brown <e****n@g****m> | over 3 years ago | |
| 96c955e9 | Lint and format | Eva Maxfield Brown <e****n@g****m> | over 3 years ago | |
| 6341a004 | Reorder the whisper segment text processing | Eva Maxfield Brown <e****n@g****m> | over 3 years ago | |
| a2ee756a | Remove special characters found in transcripts | Eva Maxfield Brown <e****n@g****m> | over 3 years ago | |
| d0979104 | Add retries to event gather and upload | Eva Maxfield Brown <e****n@g****m> | over 3 years ago | |
| d98dbc4a | Add support for py311 | Eva Maxfield Brown <e****n@g****m> | over 3 years ago | |
| eed93c44 | Force spacy not to split hypenated words | Eva Maxfield Brown <e****n@g****m> | over 3 years ago | |
| 55d51fd8 | Final v4 changes hopefully | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| e9517b46 | Remove the capitalize function in favor of str manip | evamaxfield <e****n@g****m> | almost 4 years ago | |
| 8b49d32a | Lint and format | evamaxfield <e****n@g****m> | almost 4 years ago | |
| 7f461e07 | Seemingly working, capitalize sentences | evamaxfield <e****n@g****m> | almost 4 years ago | |
| 2063b888 | Still working on debugging whisper sentencing | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| 0f849fae | Switch to MPLv2 | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| 43e3119f | Forgot to compare spacy output with the sent text | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| 8978e778 | Add retry to generate transcript due to flaky model loading | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| a6770aa7 | Lint and format | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| 67c1fb9a | Only convert once if needed, fix bug in sentence splitting | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| 5534d0b1 | Ignore lib tests for faster debugging | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| fcb89db6 | Trying to debug word and sentence matching | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| 694849f7 | Impl faster whisper | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| 4b75dcce | Separate faster-whisper from package deps | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| 1dac0e94 | Try pushing without the git commit for faster whisper | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| c999d3d1 | Pin faster-whisper to specific commit | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| 3e04a1ff | Remove test docker build | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| eccb094d | Update step docker meta id to match | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| b8080506 | Fix syntax of ci yml | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| 9358f94a | Include CI to build docker | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| b76a0727 | Use correct faster whisper branch | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| c0bfb6df | Include dockerfile, include faster whisper, fix pipeline config typing | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| eadbab0f | Transcripts are :chef-kiss: perfect with normal whisper | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| 64b8255a | Testing faster whisper | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| 5264bc42 | Lint and format | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| 1e19e6b4 | Remove any empty sentences with word metas | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| c770cfca | Fix whisper test similarity check | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| 0a800575 | Faster whisper | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| e083c32a | More logging and hopefully faster spacy | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| ff6f53c2 | Fix tests | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| a79be80f | Add more logging to whisper | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| 256f29fa | Download audio to local machine if found from URI | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| 311ee4b7 | Simpler whisper testing | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| 62b25c21 | Minor fix for process special event | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| 406cf8f1 | Include spacy transformers in deps | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| 72d78086 | Include spacy dep and transcript json in ignore manifest | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| 020f696d | Lint and format | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| 56ea9d3a | Add new example transcript to test | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| 39eab4cf | Whisper is working! | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| e4e33af5 | Lint and format | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| a5a13633 | Debugging sentence splitting | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| 6993d6c2 | Lint and format | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| c2254ef5 | Only upload local video's during processing | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| 8ad64968 | Remove parallel proc to ensure safety for GPUs | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| 03f067f0 | Enable more google APIs | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| 7bc03bb9 | Use actual pypi release of whisper | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| 532dd61c | DataclassesJSON doesn't support future annotations | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| a2f2078a | Remove prefect viz dep, try to fix tests | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| 3457f0ef | Use future annotations | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| 3c42753e | Upgrade from cookiecutter and lint and format | Eva Maxfield Brown <e****n@g****m> | almost 4 years ago | |
| 919567a6 | feature/whisper (#228) |
Eva Maxfield Brown <e****n@g****m>
Committed by: GitHub <n****y@g****m> |
almost 4 years ago | |
| 92b0484c | bugfix/unique-temp-filenames-test (#227) |
Chris Khan <c****n@g****m>
Committed by: GitHub <n****y@g****m> |
almost 4 years ago | |
| aa8ff15f | bugfix/unique-temp-filenames (#225) |
Chris Khan <c****n@g****m>
Committed by: GitHub <n****y@g****m> |
almost 4 years ago | |
| b00f378e | feature/add-video-trimming-via-transcription-ranges-to-session (#221) |
Chris Khan <c****n@g****m>
Committed by: GitHub <n****y@g****m> |
almost 4 years ago | |
| 2d080b42 | Update generate-clip requirements file | Eva Maxfield Brown <e****n@g****m> | about 4 years ago | |
| 8ea2692b | Lower GiB requested in clipping func | Eva Maxfield Brown <e****n@g****m> | about 4 years ago | |
| 5894e37e | Reduce deps and increase mem of generate-clip | Eva Maxfield Brown <e****n@g****m> | about 4 years ago | |
| 328ee6e9 | Allow all CORS requests in clipping function | Eva Maxfield Brown <e****n@g****m> | about 4 years ago | |
| 3fb35a22 | Fix imports | evamaxfield <e****n@g****m> | about 4 years ago | |
| 78eb98fe | Ignore vimeo and m3u8 tests | evamaxfield <e****n@g****m> | about 4 years ago | |
| ff96b401 | Fix imports and lints | evamaxfield <e****n@g****m> | about 4 years ago | |
| e7973ef0 | Update from cookiecutter-py-pack | evamaxfield <e****n@g****m> | about 4 years ago |