OSDev.org
https://forum.osdev.org/

unicode-org icu - Super huge icudt72l_dat!
https://forum.osdev.org/viewtopic.php?f=1&t=56519		 Page 1 of 1
Author:  AndrewAPrice [ Thu Oct 06, 2022 3:16 pm ]
Post subject:  unicode-org icu - Super huge icudt72l_dat!
Has anyone else dealt with porting https://github.com/unicode-org/icu? The stuff inside of icu4c/source/data builds a super huge icudt72l_dat.S, that when statically linked into my applications, causes the binaries to grow by 43MB! :evil:

It's read inside of source/common/udata.c in doLoadFromCommonData and openCommonData via the macro U_ICUDATA_ENTRY_POINT. I noticed the macro ICU_DATA_DIR_WINDOWS, but I'm going to have to dig into it a little more to see if I can move it to a file. And perhaps implement processes sharing the memory of files.
Author:  nexos [ Thu Oct 06, 2022 3:19 pm ]
Post subject:  Re: unicode-org icu - Super huge icudt72l_dat!
Does your OS support shared libs? If so, then ICU can be a shared lib, and that will reduce the amount of memory / disk space used.
Author:  Octocontrabass [ Thu Oct 06, 2022 3:48 pm ]
Post subject:  Re: unicode-org icu - Super huge icudt72l_dat!
Run the configure script with "--with-data-packaging=archive" to move the data to a single external file.
Author:  AndrewAPrice [ Thu Oct 06, 2022 3:54 pm ]
Post subject:  Re: unicode-org icu - Super huge icudt72l_dat!
Unfortunately I don't have support for shared libraries. For now, I'd like to load it from a file. I'll change the U_ICUDATA_ENTRY_POINT macro to be a pointer to a function that lazily loads the data file into memory, then returns a pointer to it.

(My microkernel does support shared memory between processes. For large read-only data files, perhaps I could make a service (maybe the VFS) that reads files into shared memory. I'll have to update my shared memory to support one-writer/multiple-readers.)
Author:  AndrewAPrice [ Thu Oct 06, 2022 3:55 pm ]
Post subject:  Re: unicode-org icu - Super huge icudt72l_dat!
Octocontrabass wrote:
Run the configure script with "--with-data-packaging=archive" to move the data to a single external file.


Thanks!
Author:  AndrewAPrice [ Fri Oct 07, 2022 6:16 pm ]
Post subject:  Re: unicode-org icu - Super huge icudt72l_dat!
I got it to build with the archive file. (I use a custom build system so I'm unravelling what the make rules do and reimplementing it in my system. I had to link in stubdata but then set the macro to the directory where I put the generated .dat file.)

I ported icu because it's a dependency of Skia. I actually have Skia building and drawing my UI widgets now, except text. I have an issue with fontconfig not being able to parse my fonts.conf.
Author:  AndrewAPrice [ Mon Oct 10, 2022 4:38 pm ]
Post subject:  Re: unicode-org icu - Super huge icudt72l_dat!
I've linked in the stubdata and provided the .dat archive on disk. I'm logging file accesses in my vfs server and noticed it's not being accessed.

My more pressing issue right now is speeding up fontconfig.
Page 1 of 1 All times are UTC - 6 hours
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/