Coming to grips with Microsoft’s P#

It’s often worth keeping an eye on job ads, as they can be pointers to new or popular development tools that you might not have come across before. One recent Microsoft ad was from a team working on a new language for building reliable distributed applications.

Building reliable software at scale has become increasingly important with the growth of public clouds such as Azure. They’re forcing us to rethink many of our assumptions around how we build and operate software, moving us from a world of large, monolithic applications on a single server to one where code is composed of collections of microservices that run when and where they’re needed, and we don’t need to know how many of those services are running. Observability is more important than manageability.

Introducing P

A while back Microsoft announced and open sourced a new C-like language for event-based asynchronous programming. P was intended to be a superset of C, offering domain-specific structures that would compile to C code, which you could then use as components or libraries in your C applications. Building on C ensured that P was fast and compact, ideal for real-time systems and IoT devices.

What made P different from other languages was that it built on modeling concepts, turning computer-added engineering techniques into code. You build a model in P, test and validate it, and then compile that model, skipping the steps that are normally required to convert a model into an application. The intent is for C code to be safe, reducing the risk of memory overflows and other unsafe operations.

At heart, P is a language for building asynchronous event-driven applications, and so it’s ideal for microservice-based distributed applications on any scale. It breaks your application into a series of interacting state machines, each sending and receiving events between other state machines and outside the application. It’s an approach akin to that used by the popular Akka and by Microsoft’s own Orleans actor/message framework.

Where things get interesting is with P’s capability of using the same code to validate your application as to run it, by wrapping it in a closed-system space for model checking, using what the development team call “ghost machines” to control state interactions. These ghost machines aren’t compiled and are ignored when you’re converting P to executable C code. During testing the system is examined for responsiveness, ensuring that every machine processes every event in every state. Violations are flagged, and, where necessary, P code can be used to define specific events as “deferred,” where processing can be delayed. Events can be checked for “liveness,” so that an event can’t be delayed forever.

By searching through the entire state-space of each state machine in a P program, it’s possible to be confident of its safety. Microsoft has been using P to rework key Windows drivers, considerably improving stability and increasing crash resistance.

From P to P#

Not all applications need the bare-metal approach of P and C. Microsoft has taken P-like syntax and methods and brought them to the .Net platform as P#, which has the same relationship with C# as P has to C. Built on top of .Net’s Task Parallel Library, P# has the same safety implications as P, using familiar C# constructs and familiar development tools, right down to installing as a NuGet package. Microsoft provides a set of tools to use P# in your applications: a compiler and a series of debugging tools, including a trace viewer, a race detector, and a batch tester.

You can write P# code as a pure P# application or as a C# library, using P# elements in C# code. Both methods require you to understand how P# applications are constructed, designing and implementing its state machines. Each state machine is independent, running at the same time as all the other machines in your application. They have an input queue, which feeds event handlers, which manage state transitions, before calling external methods. You can think of each machine as a single-purpose program, taking events from its queue and using their content to process simple actions. Operations are nonblocking, so a message from one machine ends up in the input queue of the target machine.

P# is an extension of C#, with new language elements, using the .Net Roslyn compiler. As a standalone system there’s no integration with Visual Studio, so if you want to get IntelliSense and refactoring, you should use it as a C# library. That does require more code, but you’re taking advantage of Visual Studio. The team is working on full integration, which should make it easier to write pure P# applications.

Building P# applications

Key elements of any P# program are events and machines. These define the messages that a program receives and the state-machines that manage them. You can think of a P# machine as working much like a C# class, with the additional definition of states. It’s important to define an initial state for each machine in your application. Each state has a set of actions, which are executed when a machine enters and exits a specific state, with entry and exit actions in separate code blocks.

For example, an entry block can create a new instance of a state machine, sending it events as required, with events declared outside the code block. The rest of the code associated with a machine is event handling, changing its state and triggering actions. Everything is asynchronous, with events read from a P# event queue and managed via the underlying Task Parallel Library.

The real advantage of P# for building distributed applications is its safe nature, with testing tools to help minimize errors. The P# Tester serializes the distributed application, using the P# scheduler to explore your code’s responses to scheduling decisions and to various inputs, with the aim of isolating bugs. As it’s designed to find nondeterministic errors, this approach simplifies the process of finding even the most obscure issue. Parallel testing options can increase the number of instances used and apply different probabilistic models and use fuzzing to increase the probability of finding errors quickly.

P# is a powerful tool for building safe, deterministic actor/message-based distributed applications. Using it inside of C# means there’s no need to learn new skills or use new development tools. It’s even compatible with Azure’s distributed computing platforms, running on top of Service Fabric or in .Net Core containers in Kubernetes.

More exciting, it’s easy to imagine it used in conjunction with the new KEDA (Kubernetes-based event-driven autoscaling), as its event model should allow P# microservices to spawn on demand. The combination of reliable distributed code and an event-driven scaling model is compelling, and one that’s well worth considering for a new generation of cloud-native applications.