Microsoft doubles down on Rust
InfoWorld ·
After many years of watching how Microsoft develops platforms and rolls out technologies to external users, one thing is clear: anything that is important to Microsoft internally quickly becomes important to anyone building on top of Windows or Azure. The transition from internal to external tooling invariably takes the form of new features in Visual Studio and Visual Studio Code . Often these capabilities have been in use inside Microsoft for years, and they’re ready for you to use immediately in your own software. We’ve seen it happen many times before, such as with the development of C# and TypeScript. Microsoft’s Tier-1 programming languages C# and TypeScript , like C++ before them, are what Microsoft internally calls its Tier-1 languages. This status rests on being backed by a complete tool chain, from editors to compilers; integration with Windows and Azure through native SDKs and optimized libraries; and compliance with Microsoft’s internal software development life-cycle requirements. When you’re shipping software for a billion users, these trappings aren’t optional extras, they’re essential. Now there’s a new member of the Tier-1 club: the Rust language . That shouldn’t be a surprise, as Azure CTO Mark Russinovich has talked about his organization’s commitment to Rust and how Rust’s memory safety features are a key component in Microsoft’s security strategy. In addition, Microsoft was a founding member of the Rust Foundation, and has made significant investments in Rust’s Windows tooling and in Rust compilers that work as part of Microsoft’s build platform. Now Rust enjoys first-class development tooling and workflows inside Microsoft. A recent post on the Rust Foundation website details how Microsoft is working to integrate Rust with the existing Microsoft Visual C++ (MSVC) platform. This approach will allow Microsoft to ensure what the blog post calls “seamless interoperability” between the languages, with Rust inheriting Visual C++ features as they’re delivered. Using Visual C++ with Rust will allow Rust to be used to build low-level Windows services, from drivers to the kernel itself. We’ve already seen that in the Windows release of Coreutils , which are a Rust re-implementation of core UNIX commands. Coreutils allow you to use certain UNIX commands inside the Windows terminal, making it easy to switch between Windows and WSL (Windows Subsystem for Linux)-hosted Linux distributions. Now that Rust has crossed the Tier-1 threshold, we can expect to see more Rust-based Windows developer tooling, where we get access to low-level functionality while avoiding memory leaks. The secret sauce: a new Rust code generator One key to delivering production-grade Windows software with Rust is a new code generation tool for Rust’s compiler , rustc. rustc is designed to use different code-generation back ends, beyond the default LLVM . There are already variants that work with GCC and with Cranelift . If you’ve not come across Cranelift, it’s the code generator of the Bytecode Alliance , used as part of the wasmtime framework. Adding a new code generator is a matter of building a tool that connects to the Rust compiler’s APIs, takes its bytecode output, generates native code, and passes it on to your choice of build pipeline. This is the role of Microsoft’s rustc_codegen_utc tooling. It’s designed to work with the existing MSVC stack, giving Rust access to a mature, well-tested, proven part of the Windows development platform without needing any changes to your code (compatibility is baked in). This links it directly to the back end of the Visual C++ compiler, a set of tools marshaled by Microsoft’s build system to work with complex projects that mix code and libraries. Microsoft calls this back end the UTC, for Universal Tuple Compiler . You won’t see that name anywhere, though; the actual code is a DLL, C2.DLL. Code is generated and delivered as .OBJ files where it can then be passed to the MSVC linker and delivered as a binary executable. This allows Rust tooling to take advantage of the decades of work Microsoft has put into both its compilers and build tooling. There’s no need for Microsoft to duplicate that work, ensuring that existing security and resilience features carry forward to the Rust compiler chain. And the resulting code is compatible with C and C++ code that has been passed through the same back end. Managing only a single compiler back end will also make it cheaper to support Rust, with no need to run two separate compiler teams that are likely to be out-of-sync with each other and with Windows SDK and kernel development. Reducing the economic impact of Rust will speed up any transition, as well as supporting what is likely to be a long-term hybrid delivery of combined Rust and C++ applications. This approach also puts Microsoft’s Rust development on a par with platforms that use GCC and LLVM compilers, which can already use the GCC and Clang back ends with Rust, leveraging the investments into Linux and macOS application development. Code developed in Rust will get the same interoperability capabilities no matter which platform you end up targeting. Microsoft Rust tools for the rest of us Unfortunately for the rest of us, rustc_codegen_utc is currently only available internally, unlike the other Rust compiler codegen tools, which are all open-source projects. Microsoft describes it as “production ready”, and it’s used by more than a hundred Microsoft Rust projects—an interesting pointer to the scale of Rust adoption by Redmond. It’s not the only Rust tool currently hidden away, as Microsoft’s complete Rust tool chain also includes its own builds of the core rustc compiler, standard libraries. That doesn’t mean that rustc_codegen_utc is not going to arrive for the rest of us. Past experience suggests that it and undoubtedly other Rust tools will come with a major update to Visual Studio or Visual Studio Code. However, the timing is always hard to predict. Even when tooling that works inside Microsoft is highly suitable for external developers, there needs to be more for it to reach the rest of the world, with language servers and IntelliSense support, as well as full integration with the Visual Studio debugger platform. There’s also more to building code outside Microsoft’s offices, where we don’t have access to their custom build tools. In discussions on the Rust compiler forums , Microsoft has indicated that it will first upstream its test and back-end infrastructure work into the wider Rust project. That implies that it may be some time before there is a public release, but also that open sourcing the project is clearly on Microsoft’s radar. Getting rustc_codegen_utc out in public is important—adoption of Rust to build third-party device drivers should help make PCs, servers, and cloud VMs more stable and reduce security risks. Get started with Rust in VS Code For now, Microsoft offers Rust tooling that integrates with Visual Studio Code and guidance for developing on Windows with Rust . To use Microsoft’s Rust language extension for VS Code, rust-analyzer , you’ll still need the MSVC C++ Build Tools to compile and build your code, along with a Microsoft-developed Windows crate that gives you direct access to Windows APIs. An automated process collects metadata from Windows APIs, ensuring that they are always up-to-date. Microsoft also provides documentation on how Windows API calls and structures are projected into Rust . It’s worth getting started now with the existing Rust tools to become familiar with Windows development in a new language. That way you can start to take advantage of Rust’s memory safety features, understanding how the language works and what changes you need to make to your programming style. Then when Microsoft starts shipping its new internal tooling for the rest of us, you can start using it to build low-level code, like drivers or libraries, that need to interoperate with Microsoft’s own C++.
After many years of watching how Microsoft develops platforms and rolls out technologies to external users, one thing is clear: anything that is important to Microsoft internally quickly becomes important to anyone building on top of Windows or Azure. The transition from internal to external tooling invariably takes the form of new features in Visual Studio and Visual Studio Code . Often these capabilities have been in use inside Microsoft for years, and they’re ready for you to use immediately in your own software. We’ve seen it happen many times before, such as with the development of C# and TypeScript. Microsoft’s Tier-1 programming languages C# and TypeScript , like C++ before them, are what Microsoft internally calls its Tier-1 languages. This status rests on being backed by a complete tool chain, from editors to compilers; integration with Windows and Azure through native SDKs and optimized libraries; and compliance with Microsoft’s internal software development life-cycle requirements. When you’re shipping software for a billion users, these trappings aren’t optional extras, they’re essential. Now there’s a new member of the Tier-1 club: the Rust language . That shouldn’t be a surprise, as Azure CTO Mark Russinovich has talked about his organization’s commitment to Rust and how Rust’s memory safety features are a key component in Microsoft’s security strategy. In addition, Microsoft was a founding member of the Rust Foundation, and has made significant investments in Rust’s Windows tooling and in Rust compilers that work as part of Microsoft’s build platform. Now Rust enjoys first-class development tooling and workflows inside Microsoft. A recent post on the Rust Foundation website details how Microsoft is working to integrate Rust with the existing Microsoft Visual C++ (MSVC) platform. This approach will allow Microsoft to ensure what the blog post calls “seamless interoperability” between the languages, with Rust inheriting Visual C++ features as they’re delivered. Using Visual C++ with Rust will allow Rust to be used to build low-level Windows services, from drivers to the kernel itself. We’ve already seen that in the Windows release of Coreutils , which are a Rust re-implementation of core UNIX commands. Coreutils allow you to use certain UNIX commands inside the Windows terminal, making it easy to switch between Windows and WSL (Windows Subsystem for Linux)-hosted Linux distributions. Now that Rust has crossed the Tier-1 threshold, we can expect to see more Rust-based Windows developer tooling, where we get access to low-level functionality while avoiding memory leaks. The secret sauce: a new Rust code generator One key to delivering production-grade Windows software with Rust is a new code generation tool for Rust’s compiler , rustc. rustc is designed to use different code-generation back ends, beyond the default LLVM . There are already variants that work with GCC and with Cranelift . If you’ve not come across Cranelift, it’s the code generator of the Bytecode Alliance , used as part of the wasmtime framework. Adding a new code generator is a matter of building a tool that connects to the Rust compiler’s APIs, takes its bytecode output, generates native code, and passes it on to your choice of build pipeline. This is the role of Microsoft’s rustc_codegen_utc tooling. It’s designed to work with the existing MSVC stack, giving Rust access to a mature, well-tested, proven part of the Windows development platform without needing any changes to your code (compatibility is baked in). This links it directly to the back end of the Visual C++ compiler, a set of tools marshaled by Microsoft’s build system to work with complex projects that mix code and libraries. Microsoft calls this back end the UTC, for Universal Tuple Compiler . You won’t see that name anywhere, though; the actual code is a DLL, C2.DLL. Code is generated and delivered as .OBJ files where it can then be passed to the MSVC linker and delivered as a binary executable. This allows Rust tooling to take advantage of the decades of work Microsoft has put into both its compilers and build tooling. There’s no need for Microsoft to duplicate that work, ensuring that existing security and resilience features carry forward to the Rust compiler chain. And the resulting code is compatible with C and C++ code that has been passed through the same back end. Managing only a single compiler back end will also make it cheaper to support Rust, with no need to run two separate compiler teams that are likely to be out-of-sync with each other and with Windows SDK and kernel development. Reducing the economic impact of Rust will speed up any transition, as well as supporting what is likely to be a long-term hybrid delivery of combined Rust and C++ applications. This approach also puts Microsoft’s Rust development on a par with platforms that use GCC and LLVM compilers, which can already use the GCC and Clang back ends with Rust, leveraging the investments into Linux and macOS application development. Code developed in Rust will get the same interoperability capabilities no matter which platform you end up targeting. Microsoft Rust tools for the rest of us Unfortunately for the rest of us, rustc_codegen_utc is currently only available internally, unlike the other Rust compiler codegen tools, which are all open-source projects. Microsoft describes it as “production ready”, and it’s used by more than a hundred Microsoft Rust projects—an interesting pointer to the scale of Rust adoption by Redmond. It’s not the only Rust tool currently hidden away, as Microsoft’s complete Rust tool chain also includes its own builds of the core rustc compiler, standard libraries. That doesn’t mean that rustc_codegen_utc is not going to arrive for the rest of us. Past experience suggests that it and undoubtedly other Rust tools will come with a major update to Visual Studio or Visual Studio Code. However, the timing is always hard to predict. Even when tooling that works inside Microsoft is highly suitable for external developers, there needs to be more for it to reach the rest of the world, with language servers and IntelliSense support, as well as full integration with the Visual Studio debugger platform. There’s also more to building code outside Microsoft’s offices, where we don’t have access to their custom build tools. In discussions on the Rust compiler forums , Microsoft has indicated that it will first upstream its test and back-end infrastructure work into the wider Rust project. That implies that it may be some time before there is a public release, but also that open sourcing the project is clearly on Microsoft’s radar. Getting rustc_codegen_utc out in public is important—adoption of Rust to build third-party device drivers should help make PCs, servers, and cloud VMs more stable and reduce security risks. Get started with Rust in VS Code For now, Microsoft offers Rust tooling that integrates with Visual Studio Code and guidance for developing on Windows with Rust . To use Microsoft’s Rust language extension for VS Code, rust-analyzer , you’ll still need the MSVC C++ Build Tools to compile and build your code, along with a Microsoft-developed Windows crate that gives you direct access to Windows APIs. An automated process collects metadata from Windows APIs, ensuring that they are always up-to-date. Microsoft also provides documentation on how Windows API calls and structures are projected into Rust . It’s worth getting started now with the existing Rust tools to become familiar with Windows development in a new language. That way you can start to take advantage of Rust’s memory safety features, understanding how the language works and what changes you need to make to your programming style. Then when Microsoft starts shipping its new internal tooling for the rest of us, you can start using it to build low-level code, like drivers or libraries, that need to interoperate with Microsoft’s own C++.