Standard Library Ideas

[leostera]

In this issue I'm collecting thoughts of the structure of a standard library. This may seem a lot larger than you'd have in OCaml/Reason to begin with, but Erlang's is actually pretty big, and includes tons of IETF RFC languages used in networking and dist. sys, so we're not actually being that crazy here.

This also doesn't mean we need to build all of these, as in most cases there exists a single canonical library in the Erlang/Elixir world that we can use for it already, so we just need to provide good bindings to it.

Libraries

• Ops: tools for operations
  • Tracer
  • Log
  • Perf
• Core: common datatypes
  • Format
  • Strings
  • Dates
  • RegExp
  • Result
  • Option
  • Bool
  • Fun
  • Atom
• Async
  • Future
  • Timers
• Collections
  • HashMap / HashSet
  • Map / Set
  • Queue/DeQue
  • List / Vector
• Num: working with numbers
  • Int / Float
  • BigInt / BigDecimal
• FileSystem: working with the file system directly
  • File
  • Path
• OS: working with the OS directly
  • Env
• Net: common networking datatypes / specs
  • HTTP
  • URI/URL
  • MIME
• CoDecs: common encoders/decoders
  • MIME
  • Base
  • CSV, XML
  • TOML / YAML
  • JSON
• Crypto / Rand
  • UUID
  • SHA
  • ...
• Typed OTP
  • Supervisor

Feel free to drop below your ideas for the standard library and if you can drop a link to where it came from 🙌🏼

[nicobao]

Maybe, the ability to run native OCaml as NIF without hassle, and safely?
I was thinking about starting an OCaml equivalent to Rustler... This could be just a third-party library though. But I can see the benefit of bundling it into the standard lib: you can market Caramel to be a faster Erlang in some ways, as the regular developer could more easily make use of NIF while using the same syntax.

[yawaramin]

I would add a decimal floating point type. Fortunately, there is one for OCaml :-) https://github.com/yawaramin/ocaml-decimal

[leostera]

@yawaramin good thinking! Thankfully Erlang/OTP has baked in bigint support -- how would you have structured your library if OCaml's int was already a big int? 🤔 Anything in particular you'd rework?

edit: I see you're using zarith for bigint stuff, so probably all of that would go away. If you're interested in helping us reuse your lib, it'd be interesting to parametrize it with a bigint implementation, so we can pass in our default Int module, and you can pass in Zarith. Otherwise we'll likely fork ocaml-decimal into ./stdlib/num/big_decimal.ml if that's okay with you :)

[leostera]

@nicobao i think its a good idea to provide a path forward to writing NIFs in OCaml as well, but we're descoping this for the time being. We can revisit it later on! 🙌🏼

[yawaramin]

Interesting, @Ostera so Erlang ints can automatically become bigints? Same as Python I guess. Let me look into functorizing it and get back to you. There are a couple of other things that might be tough to port over though, e.g. I am using mutable arrays. Does Caramel allow that?

EDIT: also GADTs

[leostera]

We may need to parametrize the functor with that too 😂 -- there's no mutable data by default in Erlang/Elixir, but you can build support for it via NIFs (natively-implemented-functions). I have done so before for a few things like memory mapped ring buffers that essentially behave as byte arrays. Performance is better, of course, but I'm yet to compare with OCaml's.

The big problem is that now you have to be aware of all the good ol' concurrency problems, because your process will move across schedulers (cpu threads) without you getting any signals for it.

And this can be the cause of segfaults and other gnarly memory unsafe ops that will take down the entire runtime.

Something to think about. We should experiment a little more!

[nicobao]

For debugging purposes, I think having an equivalent of Elixir's IO.puts and IO.inspect for generic 'a types would be neat.

[AeroNotix]

It's unclear from the discussion here whether the intention is to wholesale implement a separate Caramel standard library, with similar functionality as Erlang's - or provide FFI bindings to them.

From playing with Caramel it appears that there's quite a lot missing in the FFI bindings to Erlang standard library functions, which is very understandable for a young project. Would PRs be acceptable to add more and more in, or is there a different intention for the Erlang standard library functions/functionality?