This will not work on Unicode
Correct.
Greek and Arabic have cases as well
Who told you that?
- 0 Posts
- 4 Comments
Joined 1 year ago
Cake day: July 25th, 2023
You are not logged in. If you use a Fediverse account that is able to follow users, you can follow this user.
2·1 year agoThank you for working on this. I found
Okhslto be of great utility.
codegen-units=1,debug=true, varyingltolto = "fat"Flags Clean build time Pre-strip size Post-strip size (default) 2:31 90.8207MiB 7.3374MiB ["-Z", "gcc-ld=lld"]2:31 91.9731MiB 7.3332MiB linker = "clang"2:32 90.8207MiB 7.3375MiB linker = "clang"; fuse-ld="mold"2:31 92.1107MiB 7.3334MiB lto = "thin"Flags Clean build time Pre-strip size Post-strip size (default) 1:33 96.9630MiB 8.1695MiB ["-Z", "gcc-ld=lld"]1:32 98.3889MiB 8.1777MiB linker = "clang"1:33 96.9631MiB 8.1695MiB linker = "clang"; fuse-ld="mold"1:32 98.6903MiB 8.1797MiB lto = falseFlags Clean build time Pre-strip size Post-strip size (default) 1:32 113.5656MiB 8.0601MiB ["-Z", "gcc-ld=lld"]1:30 115.1210MiB 8.1122MiB linker = "clang"1:32 113.5656MiB 8.0602MiB linker = "clang"; fuse-ld="mold"1:31 115.4679MiB 8.0663MiB lto = "off"Flags Clean build time Pre-strip size Post-strip size (default) 1:33 113.5666MiB 8.0601MiB ["-Z", "gcc-ld=lld"]1:31 115.1231MiB 8.1122MiB linker = "clang"1:32 113.5667MiB 8.0602MiB linker = "clang"; fuse-ld="mold"1:31 115.4697MiB 8.0662MiB codegen-units=8,debug=true, varyingltolto = "fat"Flags Clean build time Pre-strip size Post-strip size (default) 2:21 104.9842MiB 7.6304MiB ["-Z", "gcc-ld=lld"]2:19 106.1436MiB 7.6264MiB linker = "clang"2:21 104.9882MiB 7.6344MiB linker = "clang"; fuse-ld="mold"2:19 106.2864MiB 7.6325MiB lto = "thin"Flags Clean build time Pre-strip size Post-strip size (default) 1:12 134.1112MiB 9.0445MiB ["-Z", "gcc-ld=lld"]1:09 136.1897MiB 9.0660MiB linker = "clang"1:12 134.1113MiB 9.0446MiB linker = "clang"; fuse-ld="mold"1:09 136.4466MiB 9.0494MiB lto = falseFlags Clean build time Pre-strip size Post-strip size (default) 1:14 158.1049MiB 9.0328MiB ["-Z", "gcc-ld=lld"]1:11 159.9998MiB 9.1129MiB linker = "clang"1:14 158.1050MiB 9.0328MiB linker = "clang"; fuse-ld="mold"1:12 160.3123MiB 9.0428MiB lto = "off"Flags Clean build time Pre-strip size Post-strip size (default) 0:57 145.9463MiB 9.4586MiB ["-Z", "gcc-ld=lld"]0:54 148.6021MiB 9.6001MiB linker = "clang"0:57 145.9464MiB 9.4587MiB linker = "clang"; fuse-ld="mold"0:55 148.8842MiB 9.4668MiB
moldappears to be similar but not faster thanlld.With the caveat that this is not a proper benchmark since:
- I didn’t measure link time alone.
- I didn’t bother running each case multiple times picking the fastest run (since I perceived the differences to be insignificant).
And a side note,
lto = falseappears to be practically useless.
I’ve been working on something similar-ish on and off.
There are three (good) solutions involving open-source models that I came across:
Vosk has the best models. But they are large. You can’t use the gigaspeech model for example (which is useful even with non-US english) to live-generate subs on many devices, because of the memory requirements. So my guess would be, whatever VLC will provide will probably suck to an extent, because it will have to be fast/lightweight enough.
What also sets vosk-api apart is that you can ask it to provide multiple alternatives (10 is usually used).
One core idea in my tool is to combine all alternatives into one text. So suppose the model predicts text to be either “… still he …” or “… silly …”. My tool can give you “… (still he|silly) …” instead of 50/50 chancing it.