The 4th issue of the Wing Inflight Magazine.
10 posts tagged with "cloud-oriented programming"
View All TagsJson Takes Flight
Picture this: you're in the zone, crafting some spectacular code like the rock star cloud developer you are. You know that to truly harness the power of the cloud ecosystem, your system needs to be distributed. After an all-night coding spree, you've created two nifty services lovingly named service1 and service2. Now, service1 must produce a message and send it to service2. So, it's time to choose a message format standard. Given that it's 2023, XML is off the table, making JSON the clear winner.
At this point, something odd happens in your development journey. You pause to figure out how to use JSON in your chosen programming language, only to discover that you need to import a library. Sure, it's manageable, but now you have to consider the implications of using this JSON library and compare it to others. Of course, you'll make the right choice, you are a rock star, after all. However, this experience leaves you wondering: with JSON being so common, shouldn't it be built into any language designed for distributed computing?
Communication Is Key
When designing a good distributed system, several key aspects come to mind, such as fault tolerance, scalability, security, high availability, and interoperability. In this section, we'll focus on interoperability — the ability of a system to work seamlessly with other systems. This means that communication between systems must adhere to not only standardized protocols but also data formats.
Wing has been tackling the complexities of distributed computing in the cloud from various angles. One of the most notable angles is the concept of inflight functions, which serve as Wing's distributed computing primitives. These are code blocks packaged up and executed on a compute platform (i.e. AWS Lambda), playing a crucial role in Wing's vision for cloud-oriented programming. As such, it's vital to enable smooth communication between these functions, and what better way to do that than with JSON!
JSON has become the de facto standard for data interchange on the web. As a minimal, text-based format that is a subset of JavaScript, JSON is easy to read and focuses on solving a single problem: data interchange. Its simplicity and widespread adoption make JSON a clear choice for incorporating into Wing, helping us achieve the vision of a truly cloud-oriented programming language.
Introducing Wing's Json
Wing has decided to treat JSON as a first-class citizen with built-in primitives. Let's take a look at a simple example:
let j = Json {
wow: "so easy"
};
// Optionally we can omit the Json keyword prefix
let j = {
wow: "so easy"
};
That's all it takes to create a JSON object in Wing. No need to import anything!
Now, let's create an inflight function that requires a Json object as input:
bring cloud;
let b = new cloud.Bucket();
let someService = new cloud.Function(inflight (jsonString: str) => {
let message = Json.parse(jsonString);
let id = message.get("id");
let content = str.fromJson(message.get("content"));
b.put("file-${id}.txt", content);
});
// Test it out
test "storing a message" {
let payload = {
id: "abc123",
content: "Hello, Json!"
};
someService.invoke(Json.stringify(payload));
// retrieve object from bucket and validate content
let storedMessage = b.get("file-${str.fromJson(payload.get("id"))}.txt");
assert(storedMessage == str.fromJson(payload.get("content")));
}
Something noteworthy happens above when we call str.fromJson()
. All fields within a Json
object are of type Json
, and Wing does not allow for implicit type conversions from Json
to other types. Thus, we must explicitly convert the Json
object to a string using the fromJson()
function, which exists for all Wing primitive types.
Let's dive a little deeper and see what else Wing's Json
type offers.
Converting To Structs
While Json
objects are great for interoperability, they are not the best for working with data in Wing. This is because Json
objects are not strongly typed. However, Wing does provide a way to convert Json
objects to structs. This conversion is done by using the fromJson()
function, which exists on all structs created in Wing.
Lets take a look at an example:
bring cloud;
// Struct declaration of a payload object
struct Payload {
id: str;
content: str;
}
// inflight function that receives json string representing a payload
let someFunction = new cloud.Function(inflight(msg: str) => {
let jsonPayload = Json.parse(msg);
// Now we can convert the json into a struct
let payload = Payload.fromJson(jsonPayload);
let content: str = payload.content;
});
The fromJson()
function will validate the Json
object at runtime and throw an error if the object does not fully match the struct definition. This is a great way to ensure that the data you are receiving is valid.
Let's add a test that will try and pass an invalid object to our inflight function:
test "json that does not match payload" {
// remember id and content are expected to be strings in our payload struct
let invalidPayload = {
id: 1,
content: false
};
someFunction.invoke(Json.stringify(invalidPayload));
}
When invoking the test above we will get the following runtime error:
│ Error: unable to parse Payload:
│ - instance.id is not of a type(s) string
│ - instance.content is not of a type(s) string
It is also worth mentioning that if the desired behavior is not to throw an exception you can also use the tryFromJson()
function
to get an optional type back.
Primitive Types
In line with JSON's ECMA-404 standard, Wing's Json
type supports the following data primitives:
let s = Json "string"; // string
let n = Json 1; // number
let b = Json true; // boolean
let a = Json [1, 2, 3]; // array
let o = Json { a: 1, b: 2 }; // object
Mutability
As with most Wing types, Json
objects are immutable. This means that once a Json
object is created, it cannot be modified. However, Wing does provide a way to create a mutable Json
object. This is done by using the keyword MutJson
instead of Json
:
let j = MutJson { a: 1, b: 2 };
// We can now modify the json object elsewhere in the code
j.set("c", 3);
Heterogeneity
If you have played with Wing much, you're probably aware that nearly all container types in Wing are homogeneous. This is pretty crucial for Wing's type system, but homogeneous data doesn't work well for interoperability between systems. Take the following data for example:
let id = "abc123";
let gpa = 3.5;
let name = "John Doe";
let completed_courses = ["CS 101", "CS 102", "CS 103"];
let active = true;
Above, we have some student data, and it's clear that the data is heterogeneous and should be represented as such between systems. This is why Wing's Json type is an exception to the rule of homogeneous containers. The following is a valid Json object in Wing:
let student = Json {
id: "abc123",
gpa: 3.5,
name: "John Doe",
completed_courses: ["CS 101", "CS 102", "CS 103"],
active: true
};
Mission Accomplished In Progress
While we are excited about the introduction of Json
into the Wing type system, we are not done yet. There are still a few features we are working on, such as removing the .get()
and .set()
syntax for just using the []
operator or .
accessor. We also have some work to do on Json
schema validation, and some other goodies which can be found in our language spec.
As always, we welcome any feedback and would love to hear from you! Come join our
community slack and share your thoughts on Wing's Json
support. As well check out and ⭐ our Github repo
Adding purge & approxSize to the queue resource
I love working with the Wing community, and one of my favorite ways to engage is by publishing good first issues which serve as approachable gateways for newcomers to get involved in developing Wing! Recently, I was looking for an issue that could serve as a friendly introduction to Wing's various layers, without being overly complex. Adding a new method to a resource seemed to be a perfect match for these criteria. I posted, as a good first issue, the implementation of the Queue.purge
method, and within just 5 minutes I decided to take on the task myself. This felt like the perfect opportunity to dive into the heart of Wing's mechanics and explore how things work behind the scenes.
Getting Started
Before diving into implementation, it was crucial for me to go through the specifications of the resource I was about to work on. I found all the necessary details in the Wing spec, outlining the method's signature, return type, and arguments.
Adding Cloud-Agnostic Interfaces
To add a method to an existing resource, I needed to modify the implementation files of the resource that are located in wingsdk/src/cloud
folder. The cloud
folder contains cloud-agnostic code, ensuring that implementations are not tied to any specific cloud provider.
Under the cloud
folder, I navigated to queue.ts
file that contains the cloud-agnostic implementation, added the relevant methods as well as comments and documentation using code suggestions from GitHub Copilot!
Cloud-Specific Implementations
Wing's support for multiple cloud providers means that each new method must be implemented for every supported provider. Now it's time to change folders like target-awscdk
, target-sim
, and others, each contain cloud-specific implementation files. I will say that target-sim
is not really a cloud provider, but it is our local simulator that runs locally and allows us to code, test, and debug very fast.
Let me give you an example. For cloud provider target-sim
, I set the length of the messages
array to zero:
public async purge(): Promise<void> {
return this.context.withTrace({
message: `Purge ().`,
activity: async () => {
this.messages.length = 0;
}
}
}
While for the tf-aws
implementation, I implemented the following:
public async purge(): Promise<void> {
const command = new PurgeQueueCommand({
QueueUrl: this.queueUrl,
});
await this.client.send(command);
}
I adjusted the code in queue.inflight.ts
as purge
is part of the inflight API. As you can see, these implementations tailored to different cloud providers were where the real magic happened.
Ensuring Quality: Writing Tests
No new functionality is complete without testing right? So I located the appropriate provider's queue.test.ts
and realized that Purge
cannot be tested unless I have a way to get the number of messages in the Queue
. So I went back to step #1 and added Queue.approxSize
as well:
For cloud provider target-sim
:
public async approxSize(): Promise<number> {
return this.context.withTrace({
message: `ApproxSize ().`,
activity: async () => {
return this.messages.length;
},
});
}
While for the tf-aws
implementation:
public async approxSize(): Promise<number> {
const command = new GetQueueAttributesCommand({
QueueUrl: this.queueUrl,
AttributeNames: ["ApproximateNumberOfMessages"],
});
const data = await this.client.send(command);
return Number.parseInt(data.Attributes?.ApproximateNumberOfMessages ?? "0");
}
Seamlessly Integrating Documentation
Okay, coding and testing behind me, now to the final step which is to align the documentation with the new method. By executing the pnpm turbo build
command, the build process integrated my comments into the documentation. This ensures that the documentation reflects the changes I have made and would appear in the API Reference section.
A Glimpse into the Journey
For the actual code, feel free to explore this (example)[https://github.com/winglang/wing/pull/1160]. Additionally, you can gain insights from a brief video that illustrates the changes made to the files:
We would love your feedback and suggestions on Wing Slack.
Let's connect!
Inflight Magazine no. 3
The 3rd issue of the Wing Inflight Magazine.
Inflight Magazine no. 2
The 2nd issue of the Wing Inflight Magazine.
Wing Your Way to Better Infrastructure with Compiler Plugins
Customizations below the abstraction line with Wing compiler plugins.
The Great Divide: Preflight vs Inflight Resource Creation ⚔️
Why does Wing let you only create resources in preflight?
Introducing `let var` and good cognitive friction
Why we are making Wing immutable by default.
Inflight Magazine no. 1
The first ever issue of the Wing Inflight Magazine.
Cloud, why so difficult? 🤷♀️
A manifesto for cloud-oriented programming.