CAF: C++ Actor Framework

Introduction CAF 0.18 is out! More than a year has passed from designing and prototyping the new messaging layer to cutting the release that implements the changes. In our announcement on breaking changes from 11/2019, we quoted performance ... [More] considerations as one of our primary motivations for implementing the new messaging layer: Over the course of the last years of developing CAF, we came to understand message as a variant-like type (…), but the implementation (…) comes at a cost. Obvious costs through virtual dispatch, but also runtime overhead for matching message handler signatures to the content of incoming messages. Further, because a message can essentially hold anything, CAF has to include a lot of meta data in each message on the wire. This increases size overhead on the wire but also makes deserializing messages costly. Prior to CAF 0.18, we relied on std::type_info and held a run-time generated table that users filled with custom types by calling actor_system_config::add_message_type prior to initializing the actor_system. With the new type ID blocks, users instead have to list all their types statically. This also comes with the benefit of allowing CAF to check whether a type was listed as a legal message type. Not calling actor_system_config::add_message_type was a common source of error, especially when introducing new message types to an existing software. With the new type ID blocks, you still have to initialize the run-time lookup table (manually or by passing the type ID to CAF_MAIN) but this is robust under refactoring: adding a type to the block also updates the initialization logic. The new type IDs in CAF allow us to reduce the amount of meta information drastically. A caf::message simply points to an array of 16-bit values that encode the run-time types of all elements. On the wire, CAF no longer prefixes a message with the full type names as string but again with a list of 16-bit integers. The new implementation for caf::message also uses a flat memory layout instead of the previous tree structure. So how much performance improvement, if any, can we expect after porting an application to 0.18? To answer this question, we have implemented a set of micro benchmarks. The full source code is available online on GitHub. The Benchmarks Our first benchmark set is called message_creation. It looks at the run-time cost of creating messages. CAF has two APIs to do this: pass all elements at once to make_message or create a message dynamically using a message_builder: BENCHMARK_F(message_creation, make_message)(benchmark::State& state) { for (auto _ : state) { auto msg = make_message(size_t{0}); benchmark::DoNotOptimize(msg); } } BENCHMARK_F(message_creation, message_builder)(benchmark::State& state) { for (auto _ : state) { message_builder mb; message msg = mb.append(size_t{0}).move_to_message(); benchmark::DoNotOptimize(msg); } } The next two benchmark sets pattern_matching and or_else measure how long it takes to dispatch a message to a behavior. The only difference between the two sets is that the former creates the behavior form a single list of lambda expressions, while the latter wraps each lambda into an individual behavior before gluing all of them together via or_else. The messages as well as the behaviors are created once during setup before running the benchmark loop. The implementation for these benchmarks involves setting up fixtures, so we omit them here for brevity. The last benchmark set is called serialization and it measures the run-time cost of saving or loading builtin and custom types. As custom types, the benchmark uses a simple POD called foo and a slightly more “advanced” type bar that includes the custom type foo as a member variable to force CAF to recursively call inspect: struct foo { int32_t a; int32_t b; }; struct bar { foo a; std::string b; }; #if CAF_VERSION >= 1800 template <typename Inspector> bool inspect(Inspector& f, foo& x) { return f.object(x).fields(f.field("a", x.a), f.field("b", x.b)); } template <typename Inspector> bool inspect(Inspector& f, bar& x) { return f.object(x).fields(f.field("a", x.a), f.field("b", x.b)); } #else // CAF_VERSION >= 1800 template <typename Inspector> typename Inspector::result_type inspect(Inspector& f, foo& x) { return f(x.a, x.b); } template <typename Inspector> typename Inspector::result_type inspect(Inspector& f, bar& x) { return f(x.a, x.b); } #endif // CAF_VERSION >= 1800 This benchmark also creates the messages once before running the benchmark loops. We omit the implementations details here also for brevity. The sources are just one click away, though. Setup Hardware in use: 3.6 GHz 10-Core Intel Core i9 64 GB 2667 MHz DDR4 CPU Caches: L1 Data 32 KiB (x10) L1 Instruction 32 KiB (x10) L2 Unified 256 KiB (x10) L3 Unified 20480 KiB (x1) Software versions: actor-framework/micro-benchmarks: b7d063b0d884ab9c210221c5d101e4a4797b2b7e actor-framework/actor-framework: CAF 0.16.5 is tag 0.16.5 CAF 0.17.6 is tag 0.17.6 CAF 0.18.0 is commit 8d27730124440c5c7422126a222b1bf2a0262e3f (slightly before the final release tag) Results Without further ado, here are the results for the micro benchmarks using the last three major release versions. All values are wall-clock time in nanoseconds as reported by Google Benchmark. Benchmark CAF 0.16.5 CAF 0.17.6 CAF 0.18 message_creation/make_message 57.3 57.3 53.5 message_creation/message_builder 258 248 339 pattern_matching/make_message 233 219 103 pattern_matching/message_builder 239 234 102 or_else/make_message 364 347 168 or_else/message_builder 348 346 167 serialization/save_foo_binary 143 136 110 serialization/save_bar_binary 174 174 127 serialization/save_msg_2int_binary 247 237 138 serialization/load_foo_binary 47.8 31.0 8.44 serialization/load_bar_binary 84.7 55.5 16.8 serialization/load_msg_2int_binary 627 566 266         Creating messages using make_message is slightly faster than before, due to the simpler (flat) data structure used internally as of 0.18. However, the same design makes the message_builder slower than before, because once all values are added to the builder, it has to allocate a new block of memory and move or copy everything into place. Selecting a matching handler from a behavior is much quicker than before with a speedup of > 2 compared to earlier versions of CAF. The serialization components also show significant performance improvements. Saving values to a binary_serializer is now 20-70% faster, because CAF has to generate less data. The biggest win comes when deserializing objects from a binary_deserializer. There is less data to read and string lookups became simple offset lookups due to the new type IDs. Remarks When browsing the micro benchmarks repository, you might notice one additional benchmark not covered here: actors/spawn_and_await. This benchmark spawns an actor that does nothing and then waits until it terminated. The values we had measured in our performance study where 41355ns, 41142ns and 42618ns for CAF 0.16.5, 0.17.6 and 0.18 respectively. This result fluctuates more than the others. We did not focus on this result, because it measures many components in CAF all at once. If you had followed our past benchmarking articles or read some of our research papers, you may also remember the benchmark suite consisting of mixed_case, actor_creation, and mailbox_performance. For the sake of completeness, we also recorded values for those. This time, all values in seconds: Benchmark CAF 0.16.5 CAF 0.17.6 CAF 0.18 mixed_case 22.8 26.8 24.2 actor_creation 2.9 1.3 0.7 mailbox_performance 34.9 35.2 33.7         Results shown in the table above are the average of ten runs. None of these benchmarks do any networking and thus no serialization. We included them mostly as a sanity check here to make sure CAF 0.18 does not introduce performance regressions. You can also find these three benchmarks (among others) online on GitHub. The exact commit used here is 383b745a4ad18b22a48166a048cb34e1bce40aaf. Conclusion CAF 0.18 introduced a new type inspection API and static type IDs for user-defined types. These two changes are most obvious when switching to the new release, because they require code changes. Overall, we are very happy with the performance we get from the new messaging layer. Keep in mind that micro-benchmark results for serialization with speedup of 2 or more do not translate to a speedup of that magnitude for a distributed CAF application. Serialization is only one fraction of the total work. Aside from any performance gains, we are most excited about new capabilities opening up as a result of the new type inspection and messaging layer. The new type inspection API passes more information to CAF (especially field names), which allows us to read and write more data formats such as JSON. Using more compact meta information on the wire and avoiding dependencies on std::type_info also allows us to tune CAF for resource-constrained devices to enable more IOT and edge computing use cases. [Less]

Today, we are more than happy to announce the first pre-release of CAF 0.18! As outlined in our previous post, this is the first time since the inception of CAF that we raise the required C++ standard version. We took this opportunity to modernize ... [More] many, many aspects of CAF. This includes the build system, which now requires CMake in version ≥ 3.4. Please take a look at the available build options in CMake directly. The most notable changes with this release are: The type inspection API received a complete overhaul. Please refer to the updated manual section for the new DSL. One of the upsides of the new inspection API is to enable the configuration framework to pick up user-defined types. So once you provide an inspect overload, you can read your types directly from a CAF configuration. This also enables more formats such as JSON and XML, although we currently ship no default serializers for such formats. All types that the application sends in a CAF message now require a type ID. This is also a breaking change. The API for assigning type IDs is an optional part of the API since version 0.17.5, but became mandatory with this release. The new type IDs require more boilerplate code than before, but they enable improved compile-time checks and faster serialization/deserialization. CAF now ships with an API for runtime metrics that applications can export directly to Prometheus. Have a look at the brand new manual section covering this feature! The library now compiles with default visibility hidden on all platforms and we finally support generating shared libraries on Windows (i.e., building .dll files). For the full list of changes, please head over to the release notes or take a look at our changelog. We encourage everyone to try the new release and look forward to any feedback (bug reports in particular). After publishing a pre-release, we no longer make changes to the API except for bugs that we cannot fix otherwise. [Less]

Switching to C++17 The last time we have dropped support for a compiler was back in 2015 when we removed support for GCC 4.7. While C++14 had several quality-of-life features, we preferred stability over what C++14 had to offer. With C++17, the ... [More] story is different. Fold expressions and if constexpr alone are game changers for CAF. That being said, we want to stay as conservative as possible when it comes to our dependencies. So, we are not rushing towards latest GCC or Clang versions. Compiler vendors have gotten a lot better at catching up to new language versions and so did package maintainers and distributors. We did a survey on the current landscape and found that all major platforms give their users easy access to C++17 by now. Microsoft has C++17 support in their Visual Studio Community Edition, Apple ships very recent Clang versions with XCode, major BSD distributions such as FreeBSD and OpenBSD come with C++17-ready versions of Clang, and, last but not least, all major Linux distributors include at least one C++17-capable compiler in their packages. On RHEL/CentOS, devtoolset gives you access to GCC 7, SLES provides access to recent GCC versions and even the oldest (still supported) LTS versions for Ubuntu and Debian have Clang 4. This makes us confident that we are not leaving many (if any) users behind if we raise our minimal required compiler version to one of: GCC ≥ 7 Clang ≥ 4 (Apple Clang ≥ 9) Visual Studio ≥ 2019. Breaking Changes With access to C++17, we are going to drop our replacement types such as caf::optional and caf::variant. Rather than slowly fading in C++17 features and deprecating individual parts one by one, we decided to make use of this opportunity to reshape CAF as a whole. Since we are breaking the API by switching to STL types and updating clumsy C++11 parts with more robust C++17 counterparts, we believe it is the right time to also apply lessons learned from previous CAF iterations. Ideally, CAF 0.18 breaks many things at once now, so that we can resume incrementally evolving CAF after that point. Messaging Over the course of the last years of developing CAF, we came to understand message as a variant-like type able to hold any tuple type used for actor messaging. The current design can play that role, but the implementation is a highly flexible container that holds arbitrary data, offers views and even enables composing messages in tree-like structures. This flexibility comes at a cost. Obvious costs through virtual dispatch, but also runtime overhead for matching message handler signatures to the content of incoming messages. Further, because a message can essentially hold anything, CAF has to include a lot of meta data in each message on the wire. This increases size overhead on the wire but also makes deserializing messages costly. Streamlining the messaging layer in CAF has the potential to improve performance significantly. First prototypes for a new messaging layer speed up the matching of messages to handlers as well as deserializing messages by a factor of 3. The downside of that design is that users have to enumerate all allowed types in the system. We did not reach full agreement whether we want to go down that route, but it would also allow CAF to better target embedded systems, as the new messaging layer would no longer require RTTI and stores less meta data. Serialization Fold expressions and other C++17 features allow us to reimplement the serializers in CAF more efficiently. Some of the optimizations are going to require changes to the class hierarchy. We are most likely going to drop the central data_processor and remove classes such as stream_serializer that CAF no longer uses internally. OpenCL A vendor-neutral API for GPGPU programming sure sounds great. Unfortunately, OpenCL did not catch on in the way we had hoped. At this point, we can call OpenCL dead and gone. There is only legacy support available and recent versions of the standard were never implemented in the first place. Consequently, we are going to drop caf_opencl. Networking Unrelated to CAF 0.18, we are working on a new networking layer in CAF. While we generally like the actor-based Broker API of caf_io, the module struggles to integrate exchangeable transport layers. Ideally, caf_openssl would not have to come as separate module. With caf_net, we are currently working on a clean slate design that gives more flexibility in deployment while at the same time improving the user experience. We based the new API on URIs, which makes is easy to select a transport via URI scheme. We are also working on name-based access to remote actors that replaces the somewhat quirky publish/remote_actor function pair of caf_io. However, caf_net is not going to replace caf_io overnight. Once the caf_net module reached a stable version, we will move it into the main repository where it eventually replaces caf_io. [Less]

For this spotlight, we dive into the world of molecular research and talk to the software team of the highly ambitious startup Lumicks at their Amsterdam HQ. Background: Lumicks Lumicks manufactures dynamic single-molecule and cell avidity ... [More] analysis equipment. Their breakthrough technologies enable visualization of molecular interactions and acoustic manipulation of biomolecules. With their products, Lumicks helps scientists to understand life to the smallest detail, which is critical for cancer research and drug development. Founded in 2014, Lumicks quickly became a success story. Today, research sites all around the world have adopted Lumicks technology: from UC Berkeley at the U.S. west coast all the way to ShanghaiTech University in far east China. The Interview CAF Team: Dear Lumicks team, thank you for taking the time speaking to us! Lumicks works hard to deliver unique insights into biological processes at a molecular level. A recent article in The Scientist speaks about using sound waves to capture cells, which is a process you use for some of your products, while other products mention optical tweezers. A bit exaggerated: How can lasers and music help us to cure cancer? Without going into too much detail, how would you explain what your products do for people like us that don’t have degrees in molecular biology? Lumicks Team: The key thing about both the lasers and the sounds waves is that they exert tiny forces. With the music during a pop concert, you can really feel those forces in your stomach. With light, it’s perhaps a bit harder to imagine: after all, you don’t experience a “thump” whenever you turn on the light, nor do you feel the sunlight pushing you back on a sunny day. Yet even light pushes against things a tiny bit, and this is actually used in the vacuum of space to drive spacecrafts with “solar sails”. At Lumicks, we harness those tiny forces to push and pull on the cells and molecules that make up your body. We can, for instance, “grab” a DNA molecule by its ends and stretch it, and then watch how other molecules land on it to repair damage on the DNA. Researchers learn invaluable things from such experiments about the processes that lead to diseases, including cancer. CAF Team: You are working with very small scales but very high resolutions. We can imagine it takes quite a lot of data processing in real-time in order to visualize the experiments and allow scientists to manipulate individual proteins or cells while studying their behavior. What are the biggest software engineering challenges that you faced during initial product development? Lumicks Team: Perhaps surprisingly, data volume is not our biggest challenge. Our sensor data comes in at hundreds of MB per second, tops. The real challenge lies in the large variety of hardware we need to talk to from the software, and orchestrating all the hardware in a coordinated way, often with strict timing requirements. We also have both a GUI and Python interface, which can run simultaneously, and so need some careful synchronisation. Plus each researcher using our software has their own specific needs for the experiments they are doing, so we need a lot of flexibility in putting the various modules together. CAF Team: A big part of software engineering is constant learning about the problem domain and iterating on a prototype until a robust solution emerges. However, we can’t iterate endlessly. Time to market is very crucial, even critical in a startup like Lumicks. What brings the actor model, and CAF in particular, to the table that helps you iterating faster and reaching a sophisticated solution in reasonable time? Lumicks Team: Our application does a lot of parallel processing and multi-threading: there is all sorts of hardware I/O running in the background, plus data processing, storage management, etc. CAF allows us to stay sane amidst all the threads: the actor framework enforces a clear model of data ownership and saves us from a lot of mutex-related headaches. Not having to worry about a whole class of subtle, hard-to-reproduce bugs is a life saver. On top of that, the availability of compile-time checking of messaging interfaces is very useful, and something where CAF has a tangible advantage over actor implementations we’ve seen in other frameworks and languages. CAF Team: How did you learn about CAF and what convinced you to use it over other frameworks? Lumicks Team: As a team we have used actors in other languages, such as LabVIEW and Erlang, and found it to be a good model for parallel processing. During the development of our control software it became clear that actors would be a great fit and and so we undertook a review of C++ frameworks. CAF came out as a clear winner and we have been using it ever since. The active development of the library was a major factor for us, as well as its flexibility and level of compile-time safety. CAF Team: How would you summarize your experience with CAF so far? What was the first version you’ve used? Were there any unexpected obstacles in adopting CAF or aha moments? Lumicks Team: CAF has been working very well for us. We started on version 0.15.5, in the Fall of 2017. We had some small issues in the beginning, mostly related to the lack of official shared library support on Windows, but otherwise have been closely tracking the official releases. We haven’t had any major unexpected obstacles, so far, but finding the Qt mix-in was a pleasant surprise. It made integrating with our UI components much easier!  CAF Team: You develop all of your software in house with an international team of software engineers. How do you introduce new team members to CAF and how would you describe the learning curve? Lumicks Team: We haven’t found the learning curve for new team members to be too challenging. As a team we have built a number of abstractions around CAF and so this helps to reduce the surface area that new team members need to learn. We also work with CAF on a single node, and use typed interfaces as much as possible to make intentions clear and explicit. Processes within the team also help. All of our code is thoroughly reviewed before merging and we do lots of internal knowledge sharing, from working together on tricky problems to fortnightly show and tell sessions on new and interesting developments. CAF Team: What could CAF do better to smooth out the learning curve and help developers being more productive? Lumicks Team: Although the CAF documentation has improved since we first started using the library, it is still an area where more work would help to smooth out the learning curve, along with some more complex examples of CAF usage. Improved tooling for debugging would also make a big impact on productivity.  CAF Team: Final question: if you were to decide what the next feature of CAF would be, what would you have in mind? Lumicks Team: One feature that would really help us as a team is better tooling to understand message flows, for example to visualize the flow of messages between actors. It would help not only to debug issues during development but also to optimise performance. We are also particularly interested in the possibilities of the new streaming API, so seeing that move out of its experimental state would also be great! CAF Team: Thank you very much for this interview! We wish you and your team all the great success in helping scientists around the world to understand—and hopefully cure!—diseases such as cancer and Alzheimer’s! Lumicks Team: Thank you! If any of your readers are interested in working with CAF to help scientists unlock new types of experiments in a small dynamic team based in Amsterdam then check out our careers page! [Less]

We are excited to announce version 0.16 of the C++ Actor Framework! Among other features and improvements, this release adds experimental support for streams. Streams in CAF are now the fourth pillar of our communication layer and complement ... [More] asnychronous messaging, publish/subscribe groups, and request/response protocols. Fundamentally, streams give you bounded queueing of pipelined messages with backpressure. This means they provide a safe channel for sending arbitrary amounts of data without worrying about overloading receivers. Of course streams are first-class citizens in CAF. Actors can have any number of active streams and can modify existing streams at runtime, for example by adding more sources or sinks to it. This gives actors a powerful new tool that required a lot of changes under the hood. Without going into too much detail, actor mailboxes now consist of several internal queues that allow seamless multiplexing of regular actor messages and stream traffic. You can find an introduction to streams with code examples in our new manual section. Expect your projects to run and compile after updating to 0.16, since this release brings no breaking changes. However, you might get warnings for deprecated member variables in actor_system_config. We have improved support for runtime configurations in CAF by adding lists, durations and dictionaries and made settings available by name instead of requiring access to member variables: // Deprecated API: auto verbosity = cfg.logger_verbosity; // New API: auto verbosity = get_or(cfg, "logger.verbosity", defaults::logger::verbosity); CAF 0.16 bundles constants unter defaults::. This may seem like strictly more overhead and typing at first, but now you can access custom options in the same way, without having to know the subtype in order to access some member variable. Your application could add the option database.host, for example, and you would access it directly from the system-wide config. No downcasting to the subtype of actor_config_server required. Got no default? Use get_if or get instead. The API is modelled after C++17’s get_if, and get, i.e., you could query a parameter as follows: auto host = get_if<std::string>(&cfg, "database.host"); if (!host) { std::cerr << "database host missing" << std::endl; // ... } else { std::cout << "connect to database host " << *host << std::endl; // ... } [Less]

We are excited to announce version 0.16 of the C++ Actor Framework! Among other features and improvements, this release adds experimental support for streams. Streams in CAF are now the fourth pillar of our communication layer and complement ... [More] asnychronous messaging, publish/subscribe groups, and request/response protocols. Fundamentally, streams give you bounded queueing of pipelined messages with backpressure. This means they provide a safe channel for sending arbitrary amounts of data without worrying about overloading receivers. Of course streams are first-class citizens in CAF. Actors can have any number of active streams and can modify existing streams at runtime, for example by adding more sources or sinks to it. This gives actors a powerful new tool that required a lot of changes under the hood. Without going into too much detail, actor mailboxes now consist of several internal queues that allow seamless multiplexing of regular actor messages and stream traffic. You can find an introduction to streams with code examples in our new manual section. Expect your projects to run and compile after updating to 0.16, since this release brings no breaking changes. However, you might get warnings for deprecated member variables in actor_system_config. We have improved support for runtime configurations in CAF by adding lists, durations and dictionaries and made settings available by name instead of requiring access to member variables: // Deprecated API: auto verbosity = cfg.logger_verbosity; // New API: auto verbosity = get_or(cfg, "logger.verbosity", defaults::logger::verbosity); CAF 0.16 bundles constants unter defaults::. This may seem like strictly more overhead and typing at first, but now you can access custom options in the same way, without having to know the subtype in order to access some member variable. Your application could add the option database.host, for example, and you would access it directly from the system-wide config. No downcasting to the subtype of actor_config_server required. Got no default? Use get_if or get instead. The API is modelled after C++17’s get_if, and get, i.e., you could query a parameter as follows: auto host = get_if<std::string>(&cfg, "database.host"); if (!host) { std::cerr << "database host missing" << std::endl; // ... } else { std::cout << "connect to database host " << *host << std::endl; // ... } [Less]

Version 0.15.1 of CAF has just released. This patch removes the non-null guarantee on actor handles. While the non-null guarantee is a nice property and allows for cleaner code in some parts in CAF, the absence of a default constructor puts obstacles ... [More] in the way of developers trying to port from earlier version of CAF. Also, using actor handles in containers or optional was painful. Sending messages to invalid handles is a no-op, no checking by the user required. Further changes: Fix handling of integer types in actor_system_config for custom parameters Fix a possible race when spawning function-based actors Avoid ambiguous overload with inspect() when serializing/deserializing Deprecate .unsafe() (simply use bool conversion instead) Fix spawn_in_group Fix a bug that matched any atom when querying type-erased tuple directly Remove all try/catch blocks when compiling CAF with CAF_NO_EXCEPTIONS Mark optional's move ctor/assignment noexcept if noexcept in T [Less]

Version 0.15.1 of CAF has just released. This patch removes the non-null guarantee on actor handles. While the non-null guarantee is a nice property and allows for cleaner code in some parts in CAF, the absence of a default constructor puts obstacles ... [More] in the way of developers trying to port from earlier version of CAF. Also, using actor handles in containers or optional was painful. Sending messages to invalid handles is a no-op, no checking by the user required. Further changes: Fix handling of integer types in actor_system_config for custom parameters Fix a possible race when spawning function-based actors Avoid ambiguous overload with inspect() when serializing/deserializing Deprecate .unsafe() (simply use bool conversion instead) Fix spawn_in_group Fix a bug that matched any atom when querying type-erased tuple directly Remove all try/catch blocks when compiling CAF with CAF_NO_EXCEPTIONS Mark optional’s move ctor/assignment noexcept if noexcept in T [Less]

Version 0.15.1 of CAF has just released. This patch removes the non-null guarantee on actor handles. While the non-null guarantee is a nice property and allows for cleaner code in some parts in CAF, the absence of a default constructor puts obstacles ... [More] in the way of developers trying to port from earlier version of CAF. Also, using actor handles in containers or optional was painful. Sending messages to invalid handles is a no-op, no checking by the user required. Further changes: Fix handling of integer types in actor_system_config for custom parameters Fix a possible race when spawning function-based actors Avoid ambiguous overload with inspect() when serializing/deserializing Deprecate .unsafe() (simply use bool conversion instead) Fix spawn_in_group Fix a bug that matched any atom when querying type-erased tuple directly Remove all try/catch blocks when compiling CAF with CAF_NO_EXCEPTIONS Mark optional’s move ctor/assignment noexcept if noexcept in T [Less]

Just a quick announcement: CAF 0.15 is finally out! We delayed the release due to two last minute bug fixes. With 0.15 out in the wild, we can start working on exciting new features again.