UnifontEX: My programming font that took 6 years.

stgiga@lemmy.blahaj.zone to Transprogrammer@lemmy.blahaj.zone – 34 points –
stgiga.github.io

6 years ago I set out to improve GNU Unifont, and finally after 6 years I have finished. It has MANY special Unicode symbols, including gender ones and plenty of technical ones. I use it as my IDE and terminal fonts on ALL my OSes. Oh and this time I fixed the link.

Also, "UnifontExMono.png" is both its own preview image as well as a proper build of the font for use cases where TTF and BDF are too big, like in character LCDs. I also do extensive documentation of my content so don't hate me.

Here's a link: UnifontEX

Logo: UnifontEX's logo

28

Do you have an extended preview image? The .png preview on your GitHub seems to just be black.

I didn't make a preview image.

Y'know, it's kinda important to know how a font looks like, that's pretty much the whole point.

Actually, technically speaking, that PNG is a preview image. It's ALL the characters of the font laid out in their Unicode positions in 16x16 cells from U+0000 to U+10FFFF

Also I was asleep when Luna replied, and I was really tired when I said I didn't make a preview image. I actually DID, but it's the TTF2PNG build of the font that was generated in a special way that makes it so that the image can be read sequentially from U+0000 to U+10FFFF in 16x16 cells to get the character you want, with no need for a definition file that shows where a codepoint is in the image. Also, it means that it also serves as a 1:1 preview that is properly mapped too.

As for why the image is "black", it has to do with the fact that to store U+0000 to U+10FFFF in 16x16 cells in a way compatible with sequential reading it requires that the sheet be 4096x65536. Blink engine browsers as well as Samsung's Gallery app have no problems viewing it (as well as some others), but there are also quite a few viewers that really hate this size. Oh, and the PNG is in 1bpp Indexed Color mode to get it small enough to fit in the 1MiB figure used by the 3DO console's font chip size (Apparently the 3DO had the largest font chip.) Basically, the TTF2PNG build is its own preview image, but it may be difficult to display on some viewers. Sorry for any confusion.

Please don't hate me.

Blink engine browsers as well as Samsung’s Gallery app have no problems viewing it (as well as some others), but there are also quite a few viewers that really hate this size.

Firefox wasn't doing good with displaying that.

Please don’t hate me.

No need to hate you for this.

I think some images would go a long way to help the text.

What is this, then?

It's ALL the characters of the font laid out in their Unicode positions in 16x16 cells from U+0000 to U+10FFFF

Actually, technically speaking, that PNG is a preview image. It's ALL the characters of the font laid out in their Unicode positions in 16x16 cells from U+0000 to U+10FFFF

Also I was asleep when ChaoticNeutralCzech replied, and I was really tired when I said I didn't make a preview image. I actually DID, but it's the TTF2PNG build of the font that was generated in a special way that makes it so that the image can be read sequentially from U+0000 to U+10FFFF in 16x16 cells to get the character you want, with no need for a definition file that shows where a codepoint is in the image. Also, it means that it also serves as a 1:1 preview that is properly mapped too.

As for why the image is "black", it has to do with the fact that to store U+0000 to U+10FFFF in 16x16 cells in a way compatible with sequential reading it requires that the sheet be 4096x65536. Blink engine browsers as well as Samsung's Gallery app have no problems viewing it (as well as some others), but there are also quite a few viewers that really hate this size. Oh, and the PNG is in 1bpp Indexed Color mode to get it small enough to fit in the 1MiB figure used by the 3DO console's font chip size (Apparently the 3DO had the largest font chip.) Basically, the TTF2PNG build is its own preview image, but it may be difficult to display on some viewers. Sorry for any confusion.

Please don't hate me.

Your explanation is OK, I just don't know why you've mentioned the 3DO. Most viewers are OK with giant images as long as the file size is not too big; my guess for the image being black would be that the background is transparent and some viewers render it on a black background but I think it's actually white. And of course the image is 1bpp because the font is in a 16x16 monochrome raster.

As for why I brought up the 3DO: It's because the TTF2PNG version is just the right size (1MiB, thanks to the 3DO's precedent) to be a retro Unicode font ROM in various old computers and devices. I've even thought about making a version of FreeDOS modeled after DOS/V (a version of MS-DOS made to display Japanese and Korean characters on VGA DOS machines) using it, and I call it "DOS/U" (Unicode DOS). I've also planned on using it in an open-source fantasy retro computer I make as the video chip's internal font, and the audio chip is something else I have planned out too.

Idk, the texture cache is usually not compressed and often you can't choose the bit depth. I don’t think 3DO games had thousands of monochrome characters, more likely a small range of nicely drawn ones – stylized, antialiased, proportional and even shaded.

Not to downplay your work of 6 years but your font might fit in a terminal (which is what Unifont was designed for) or perhaps a late 1990s phone in the Asian market (where emoji originated, after all).

When making a fantasy 3D console, you may want to consider modern graphics cards’ shader pipelines and restrict the resolution, FLOPS and memory to make writing games and/or emulators possible in standard libs like OpenGL. Also, a physical incarnation of the console will not need complicated FPGAs to run efficiently, just a cheap or old PC GPU.

I've seen what the 3DO BIOS font looks like, and it's DEFINITELY not 16px. But it isn't a color font, per se. Also, I'm not targeting the 3DO per se, or any specific platform. I only mentioned the 3DO because historically it was the one device where the Kanji ROM (it's in Western units too though. Also, Kanji ROMs were common things in the olden days of console and PC gaming [1980s and early 1990s]) is 1MiB. I haven't seen larger. Also, note that there are hardware DEFLATE decoder chips. Also, note that 4096x65536 1bpp uncompressed would be 32MiB. Also, the original PK-ZIP could even run on the 4.77MHz 8088 1981 IBM PC. DOS/V requires a VGA card, which tended to belong to FAR faster machines than that. So, real-time decompression in software for DOS/U isn't a problem. Also, the GBC is clocked far higher than the 8088. And there was one GBC game that went to 8MiB and had FMV. Pokemon Crystal only went to 2MiB. Technically speaking, you could put the TTF2PNG version on the last eighth of the cartridge, and do Unicode fan translations of any game that works on MBC5 mappers (or can be ported to it. Pokemon Crystal's bootleg Chinese fan translation used 16px characters, as did the Star Beasts Pokemon clone, so 16x16 characters can be displayed.)

Now the terminal and phone ideas are ALSO possible. I mean, Unicode DOES have a LOT of OS and technical symbols, as does UnifontEX. In fact, UnifontEX having BOTH "Mathematical Alphanumeric Symbols" AND "Letterlike Symbols" together makes the Fraktur escape code and the Bold code + Fraktur code possible. It also makes support for the earlier emoji (the stuff that was able to fit in Plane 0 Unicode block gaps, and which had the most attachment to the original cell phone sets) possible in conjunction with more recent (2018 and before) ones. Also, my fantasy console wasn't really going to be 3D per se. Basically, it would be a fantasy computer designed to be a better Commander X16. It would have a 50MHz eZ80 (equivalent to a hypothetical 200MHz regular Z80), a full-on GS ROMpler as a soundchip (except using my JummBox SoundFont for the samples, stored in Section 11 of the SoundFont specification's Silicon SoundFonts mode, a mode made for making ROMplers out of SoundFonts, and that SoundFont fits the bill because it is very very close to an actual chip size. It's 0.99GiB, but more specifically, 1020.9MiB. So yes, the sound ROM would be 1GiB, but in 7z it is only 198MiB, but Silicon SoundFonts is happiest with a round number size and direct loading, so thus we end up with a 1MiB font chip in retro style, and a 1GiB sound BIOS in retro style), which I can't decide between making the 64ch that the Atari AMY would have had (64 additive sines), or the 128ch of the last GS and XG MIDI synths, which is also equivalent to adding up the real chip equivalents of some parts of the SoundFont, so I'm probably just going to do a toggle. Also I would make the video modes extremely overkill, basically: Sharp X68000 video modes with Amiga AGA HAM8 color detail across the board.

As for terminals: I use UnifontEX in ALL my terminal apps and text editors. Putting it into a hardware terminal would be a sterling idea. Also, if we are going to do that, it's also an excellent idea to put it into a TDD (used by people who cannot hear or who have limited hearing to communicate over telephone lines), to make them more comprehensive in their character support, which is helpful if languages of a historic nature are involved, and the sheer amount of technical symbols could make tech support a lot easier, because you could show the symbols. Also, Unicode's musical notation is handy here too, because one could send musical notation over a TDD. Also, Plane 1 of Unicode has playing cards (including tarot), dominoes, and Mahjong. Oh, and the huge amount of math symbols in UnifontEX is partially what drew me to it, because I used a very early ancestor of it in my Physics Honors class in high school to input the various symbols used in physics via Microsoft Word's character insert function. Yes, UnifontEX was something I started around my early years of high school. I'm now 21 with a degree in cybersecurity.

With regards to mobile phones: I also say yes, because honestly it can help with texts (Also, this supports emoji from 2018 and before, which coincidentally was the heyday of Tumblr, which is one of the easiest places to find emoji pronouns. Note that some of my pronouns ARE Unicode and are in UnifontEX, so this isn't a judgement. So, if we factor that in, I can say that a significant chunk of emoji pronouns are in UnifontEX. As is the nonbinary symbol, U+1F72C, and some other characters in that block seem to represent some other less-well-known gender symbols. As well as characters in the Miscellaneous Symbols and the Miscellaneous Symbols and Arrows blocks. Basically, having both planes in one font allows for high inclusivity. Also, you have a lot more ability to send technical information over text. I seriously believe that UnifontEX should be usable on mobile as a fallback, since both iOS and Android have no problem displaying it at any scale I've tried.) Also, the DoCoMo 1999 emoji were 12x12. The first mobile emoji set was the SoftBank SkyWalker set from 1997, which had 32x32 monochrome emoji. UnifontEX's emoji are 8x16 when they can fit (the basic smiley and frown are based off of MS-DOS's 8px ones), but the vast majority of emoji in UnifontEX are 16x16px. The "Emoticons" block in Plane 1 which has all the face emoji is special because in UnifontEX, it ALL looks like monochrome versions of the 16x16 forum smilies. So seeing them on a phone of that era is reasonable. Oh also, Inkbox on YouTube made videos where he ported Unifont's Chinese characters to the Apple II and Commander X16. Also, UnifontEX's pop culture references don't stop at forum smilies. The ROFL character is a face with ROFL over the top of it because the diagonals would be a problem otherwise. The ghost emoji is a monochrome version of the Pac-Man ghost (I didn't make that decision), the Korean characters are based on Galmuri Gothic (the maker of Unifont 15.0.06's Hangul was the creator of Galmuri Gothic), which is modeled a bit after the Nintendo DS Korean fonts. And some of the emoji's faces are literally like the MS-DOS smiley face. So, UnifontEX's components were influenced by pop culture to some degree, but not in a bad way. Oh fun fact: The type of sans-serif that Unifont is is actually the same type as both versions of the Discord message font. If you use Firefox and set it to force ALL page fonts on ALL websites to UnifontEX as I have done, and then visit browser Discord, it actually looks fine. Note that Macs will have better non-integral scaling, but if your Windows machine has a 3840-wide display then there is no issue there either. My 1080p Linux and Windows machines have trouble. What I honestly want is for sites such as Archive Of Our Own and other sites with literature available online to have a UnifontEX mode (obviously at 16px or integer multiples of it) just in case people use rarer Unicode, or want to, in their stories. In fact, I want UnifontEX's portion of Unicode to be considered an agreed-upon subset of it as a successor to WGL, as well as for various sites. Also, I want it to be agreed-upon for Unicode art, similar to how Mona/MS PGothic is for Japanese Shift_JIS art. Another use would be as a minimum character set for captioning devices.

TL;DR: there's no texture cache involved.

Why cap emoji support at 2018 instead of drawing the 150 characters? (If you want, I’ll draw them for you when I have free time in early October.) 🥻🩵🦻🧑‍🦽 They feature cultural clothing, people with disabilities and more colored squares & hearts to make pride flags out of.

Anyway, what are those MOGAI & Tumblr pride flag & pronoun characters’ codes? Private use area? Is there a standard or just a first-come-first-served basis?

I mean, the base form (substantivum) of my Czech pronoun kinda is in Unicode (🔛) but nobody would understand what I mean if I tagged myself with it. Also, nobody ever “asks for pronouns” in Czech (my answer would be on/něj/jemu/ho/-/něm/ním but still would not cover all Czech grammatical gender shenanigans like adjective/verb endings) so Czechs just have to pick one of the existing ones (♂️/♀️ only), which is not ideal but at least they have a standard Unicode representation.

It's not that I'm purposefully capping it at 2018. It's that TrueType and OpenType only allow 65535 glyphs, and I'm at 65417. Also, Unifont developers have drawn the relevant glyphs (they've drawn even Unicode 15 Plane 1 and still haven't stopped) but they aren't going to fit, even though they are compatible. I really want to put them in, but the old TrueType/OpenType format is preventing me from doing this. If you want me to put them in, please talk to the OS vendors to utilize Apple's old iOS SVG webfont format which isn't limited to 65535 glyphs. I can't even use versions of Unifont Upper 11 higher than 11.0.01 due to that same 65535 glyph count limit. Also Unifont is not color, and nor is UnifontEX. The trans flag in the retro computer BIOS would be handled by looking for those combining characters and drawing the trans flag in a 16x16 cell using color bars. Oh, and I'm not purposefully capping Upper at 2018. Also, the Tumblr codes aren't in the PUA and either use existing characters or KreativeKorp's Vexillo, which does it via tag characters, which the OS would also look for and fill in in a similar way. Of course, Unifont and UnifontEX's architecture isn't one that handles any sort of joined characters. Any emoji that use ZWJ sequences are always rendered as their components, and this is unfixable. Not to mention that one would run into the 65535 glyph issue pretty much instantly. So, I absolutely agree with your idea, but I unfortunately cannot implement it due to multiple technical limitations. Sorry about that. Also, with regards to the emoji pronouns, the first example that came to mind would be ones like 🐰/🐰s or such.

TL;DR: I absolutely agree with your comment but multiple technical limitations prevent implementation of it.

Whoops, it makes sense now. I did not realize the 3DO needs a BIOS (of course it does, it has to boot from a CD). Yeah, Unifont⅀𝕏 is a fine BIOS font, and I like when a fantasy console like the PICO-8 boots into some environment when no game is inserted. Make sure the most important combining sequences like 🏳️‍⚧️ U+1F3F3+FE0F+0200D+26A7+FE0F are properly supported!

The 3DO's BIOS font alone is 1MiB, which is why the file size matters here. UnifontEX-on-a-chip is historically-accurate in terms of file size on 1990s devices (the 3DO Blaster card was literally a 3DO on an ISA card for DOS and DOS/V machines), and in character size (The NEC PC-98, all DOS/V installs, Fujitsu FM Towns, Sharp X68000, and MANY other Japanese computers of the 1980s and 1990s used 16x16 for fullwidth characters and 8x16 for halfwidth characters). So, ignoring the stuff like the Bitcoin symbol, UnifontEX could have existed in an early-1990s console or PC, or upgrade board for a late 1980s machine like the MSX, which had swappable Kanji ROM chips and the needed resolution. Well, rather than just Kanji, how about a decent chunk of Unicode? Also, your comment about late 1990s mobile phones isn't far off. Unicode did exist in the 1990s. Ultimately, UnifontEX's PNG form is historically-accurate given the 3DO's file size. It's why having a 1MiB version is so crucial, because it is within the file size range of Kanji ROMs from the olden days. It's basically a Unicode ROM as well as its own preview image. Think of all the things you can do. You could put it into character LCDs and so many other things. I'm currently tired after a long day of college so I'm not able to go on as long as I usually do.

Also, I'd handle the trans flag sequence in that OS as a one-character (fullwidth) color bar. Also, the environment that would be booted into would be a computing environment, drawn as much as possible in UnifontEX glyphs. A form of super-textmode. But graphics mode would not conflict either. Note that UnifontEX has MANY gender symbols, including ones you would see in the MOGAI community as well as a third flag (U+26FF, the Rumpus Parable agender one). Also, we are dealing with monochrome 16x16 pixel here, so the flags are done in the graphics renderer not the font. I'm literally falling asleep at my keyboard right now and can't think further on this topic, so I will stop here.

1 more...
1 more...
1 more...

Incredibly based. Love the rant-like structure of the post rsther than boring bullet points with pretty badges. I don't need a sales pitch to understand why I need to use this font.

Did you mean to link to the project? Maybe it's my app but I don't see anything.

The most prominent image on the page should appear as the link preview so adding the logo link to README.md could help; you can add an image and/or link to the repo in the post body.

I did do the Markdown image embed into the README, as well as a repo link and image in the body of the post, but regrettably Lemmy isn't making the actual post image work.

I just made a NEC PC-98 version via Neko Project 2 (the Wii version can make TrueType fonts into ANEX86.BMP replacements for PC-98 emulators) which has been uploaded.

UPDATE:

UnifontEX has now received a JSTF table, a process that fixed compatibility with some programs. It also has been given a vector DFONT and a better WOFF1 in terms of bandwith. PLEASE redownload.