Skip to content

Use Boost.Charconv for double serialization - #993

Draft
grisumbras wants to merge 1 commit into
boostorg:developfrom
grisumbras:refactor/charconv
Draft

Use Boost.Charconv for double serialization#993
grisumbras wants to merge 1 commit into
boostorg:developfrom
grisumbras:refactor/charconv

Conversation

@grisumbras

@grisumbras grisumbras commented Mar 19, 2024

Copy link
Copy Markdown
Member

Removes embedded version of ryu and uses Charconv for double formatting.

@cppalliance-bot

Copy link
Copy Markdown

@cppalliance-bot

Copy link
Copy Markdown

@cppalliance-bot

Copy link
Copy Markdown

An automated preview of the documentation is available at https://993.json.prtest2.cppalliance.org/libs/json/doc/html/index.html

@cppalliance-bot

Copy link
Copy Markdown

An automated preview of the documentation is available at https://993.json.prtest2.cppalliance.org/libs/json/doc/html/index.html

1 similar comment
@cppalliance-bot

Copy link
Copy Markdown

An automated preview of the documentation is available at https://993.json.prtest2.cppalliance.org/libs/json/doc/html/index.html

@grisumbras
grisumbras force-pushed the refactor/charconv branch 2 times, most recently from 227539e to be9028c Compare December 14, 2024 17:06
@cppalliance-bot

Copy link
Copy Markdown

An automated preview of the documentation is available at https://993.json.prtest2.cppalliance.org/libs/json/doc/html/index.html

1 similar comment
@cppalliance-bot

Copy link
Copy Markdown

An automated preview of the documentation is available at https://993.json.prtest2.cppalliance.org/libs/json/doc/html/index.html

@codecov

codecov Bot commented Dec 14, 2024

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.71%. Comparing base (ccdd27e) to head (028d69b).
⚠️ Report is 1 commits behind head on develop.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           develop     #993      +/-   ##
===========================================
- Coverage    93.91%   93.71%   -0.21%     
===========================================
  Files           91       85       -6     
  Lines         9288     8971     -317     
===========================================
- Hits          8723     8407     -316     
+ Misses         565      564       -1     
Files with missing lines Coverage Δ
include/boost/json/detail/impl/format.ipp 100.00% <ø> (ø)
include/boost/json/impl/serializer.ipp 100.00% <100.00%> (ø)

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ccdd27e...028d69b. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@cppalliance-bot

Copy link
Copy Markdown

@cppalliance-bot

Copy link
Copy Markdown

An automated preview of the documentation is available at https://993.json.prtest2.cppalliance.org/libs/json/doc/html/index.html

@cppalliance-bot

Copy link
Copy Markdown

@cppalliance-bot

Copy link
Copy Markdown

An automated preview of the documentation is available at https://993.json.prtest2.cppalliance.org/libs/json/doc/html/index.html

@grisumbras
grisumbras force-pushed the refactor/charconv branch 2 times, most recently from 91a7792 to 9e788cd Compare February 19, 2025 11:11
@cppalliance-bot

Copy link
Copy Markdown

An automated preview of the documentation is available at https://993.json.prtest2.cppalliance.org/libs/json/doc/html/index.html

@grisumbras
grisumbras force-pushed the refactor/charconv branch 2 times, most recently from 3b7526f to c58ed7b Compare February 19, 2025 11:17
@cppalliance-bot

Copy link
Copy Markdown

An automated preview of the documentation is available at https://993.json.prtest2.cppalliance.org/libs/json/doc/html/index.html

2 similar comments
@cppalliance-bot

Copy link
Copy Markdown

An automated preview of the documentation is available at https://993.json.prtest2.cppalliance.org/libs/json/doc/html/index.html

@cppalliance-bot

Copy link
Copy Markdown

An automated preview of the documentation is available at https://993.json.prtest2.cppalliance.org/libs/json/doc/html/index.html

@cppalliance-bot

Copy link
Copy Markdown

@cppalliance-bot

Copy link
Copy Markdown

@cppalliance-bot

Copy link
Copy Markdown

An automated preview of the documentation is available at https://993.json.prtest2.cppalliance.org/libs/json/doc/html/index.html

@cppalliance-bot

Copy link
Copy Markdown

@sdarwin

sdarwin commented Feb 19, 2025

Copy link
Copy Markdown
Collaborator

This might be a problem with Boost.Charconv.

It used more than 8GB of memory.

I have just kicked off another run on 'develop', to compare that.

@sdarwin

sdarwin commented Feb 19, 2025

Copy link
Copy Markdown
Collaborator

Depending on the test, even the develop branch may get to nearly 8GB.
Charconv is not the cause.
I don't think JSON was using that much memory a couple years ago.
In any case the runner is now parallel-jobs=1 so more memory is available.

@grisumbras

Copy link
Copy Markdown
Member Author

I've recorded memory usage when building and running tests. The outlier is pretty much compiling parse_into.cpp:

Config Max mem usage
toolset=gcc variant=release cxxstd=20 3.22GB
toolset=gcc variant=release cxxstd=17 3.15GB
toolset=gcc variant=debug cxxstd=20 3.10GB
toolset=gcc variant=debug cxxstd=17 2.95GB
toolset=clang variant=debug cxxstd=20 2.02GB
toolset=clang variant=debug cxxstd=17 2.00GB
toolset=clang variant=release cxxstd=20 1.51GB
toolset=clang variant=release cxxstd=17 1.42GB

Everthing else is below 0.9GB (https://gist.github.com/grisumbras/5985960c7781c4f657a93605d9e2b0ed).

Unfortuantely, this is inherent to parse_into.cpp. It's very template-heavy. I think, the way we should deal with this is to move it to a separate build target, and run b2 twice: one for regular jobs with regular job count limit, and then the heavy tests one job at a time.

@grisumbras

Copy link
Copy Markdown
Member Author

BTW, for posterity, the info was collected thusly:

# user-config.jam
using gcc : 13 : /usr/bin/time "-f%C|%M" -o/home/grisumbras/dev/stats -a g++-13 ;
using clang : 20 : /usr/bin/time "-f%C|%M" -o/home/grisumbras/dev/stats -a clang++-20 ;

And then b2 was invoked with

rm /home/grisumbras/dev/stats
b2 test example toolset=gcc,clang variant=debug,release cxxstd=17,20 warnings=extra testing.launcher="/usr/bin/time -f'%C|%M' -o/home/grisumbras/dev/stats -a"

After that the file /home/grisumbras/dev/stats contained collected statistics.

@cppalliance-bot

Copy link
Copy Markdown

An automated preview of the documentation is available at https://993.json.prtest2.cppalliance.org/libs/json/doc/html/index.html

@cppalliance-bot

Copy link
Copy Markdown

@cppalliance-bot

Copy link
Copy Markdown

An automated preview of the documentation is available at https://993.json.prtest2.cppalliance.org/libs/json/doc/html/index.html

@cppalliance-bot

Copy link
Copy Markdown

@cppalliance-bot

Copy link
Copy Markdown

An automated preview of the documentation is available at https://993.json.prtest2.cppalliance.org/libs/json/doc/html/index.html

@cppalliance-bot

Copy link
Copy Markdown

@cppalliance-bot

Copy link
Copy Markdown

An automated preview of the documentation is available at https://993.json.prtest2.cppalliance.org/libs/json/doc/html/index.html

@cppalliance-bot

Copy link
Copy Markdown

@cppalliance-bot

Copy link
Copy Markdown

An automated preview of the documentation is available at https://993.json.prtest2.cppalliance.org/libs/json/doc/html/index.html

@cppalliance-bot

Copy link
Copy Markdown

@grisumbras grisumbras changed the title Use Boost.Charconv Use Boost.Charconv for double serialization Aug 12, 2026
@cppalliance-bot

cppalliance-bot commented Aug 12, 2026

Copy link
Copy Markdown

An automated preview of the documentation is available at https://993.json.prtest2.cppalliance.org/libs/json/doc/html/index.html

If more commits are pushed to the pull request, the docs will rebuild at the same URL.

2026-08-12 16:15:17 UTC

@cppalliance-bot

Copy link
Copy Markdown

GCOVR code coverage report https://993.json.prtest2.cppalliance.org/gcovr/index.html
LCOV code coverage report https://993.json.prtest2.cppalliance.org/genhtml/index.html
Coverage Diff Report https://993.json.prtest2.cppalliance.org/diff-report/index.html

Build time: 2026-08-12 16:22:40 UTC

@cppalliance-bot

Copy link
Copy Markdown

GCOVR code coverage report https://993.json.prtest2.cppalliance.org/gcovr/index.html
LCOV code coverage report https://993.json.prtest2.cppalliance.org/genhtml/index.html
Coverage Diff Report https://993.json.prtest2.cppalliance.org/diff-report/index.html

Build time: 2026-08-12 16:24:42 UTC

@cppalliance-bot

Copy link
Copy Markdown

@cppalliance-bot

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants