Watermill v0.2.0 released

Let me start by thanking all contributors for feedback on Watermill - it drives us to add new features. Thanks!

It’s been almost a month since the initial release of Watermill. However, it’s just the beginning and we are still working hard to ship new features.

What is new in Watermill 0.2?

Documentation - watermill.io

Godoc is great. However, it’s functionality is sometimes too limited to express more complicated documentation.

For that reason, we’ve created watermill.io.

The most important parts of the documentation are:

• Getting Started guide which will explain you the most basic concepts of the Watermill by the examples.
• Pub/Sub’s implementations which contains detailed documentation of all built-in Pub/Sub’s implementations.
• Router documentation which explains how Messages Router is working.
• All documentation topics

Of course, we are not abandoning godoc. You can still find all important information in the comments (in fact, a large part of watermill.io is generated directly from godocs).`

Added context.Context to the message.

From now on, the Message holds a Context, which carries deadlines, cancelation signals, and other message-scoped values.

Unlike metadata, it is not serialized nor sent to the Pub/Sub.

Replaced Kafka Pub/Sub implementation with Sarama

Watermill development started when Sarama didn’t have support for consumer groups yet. Fortunately, it is now available. Thanks to this fact we could replace librdkafka implementation with Sarama.

Using Sarama allowed us to get entirely rid of the cgo dependency in Watermill. Another great bonus is a great improvement of messages publishing performance (quick benchmarks showed upgrade from 10k messages/s to 75k messages/s).

Because the client implementation has been replaced, this change is not backward compatible.

Kafka Breaking changes and API cleanups:

• kafka.NewCustomPublisher has been removed, please use NewPublisher instead.
• kafka.ConfluentConsumerConstructor has been removed and is no longer needed.
• kafka.SubscriberConfig.NoConsumerGroup is no longer needed. You can now just pass empty kafka.SubscriberConfig.ConsumerGroup.
• kafka.AutoOffsetReset has been removed. Please use github.com/Shopify/sarama.Config.Consumer.Offsets.Initial passed to NewSubscriber overwriteSaramaConfig argument instead.
• kafka.ConsumersCount has been removed and is no longer needed.
• kafka.KafkaConfigOverwrite has been removed and is no longer needed. You can now pass sarama.Config to NewSubscriber and NewPublisher.
• kafka.NewConfluentSubscriber has been removed. Please use kafka.NewSubscriber instead.
• kafka.NewCustomConfluentSubscriber has been removed. Please use kafka.NewSubscriber instead.
• kafka.DefaultConfluentConsumerConstructor has been removed and is no longer needed.
• kafka.Marshaler and kafka.Unmarshaler interfaces have been changed to be compatible with the Sarama API.

Google Cloud Pub/Sub

Cloud Pub/Sub brings the flexibility and reliability of enterprise message-oriented middleware to the cloud. At the same time, Cloud Pub/Sub is scalable, durable event ingestion and delivery system that serves as a foundation for modern stream analytics pipelines. By providing many-to-many, asynchronous messaging that decouples senders and receivers, it allows for secure and highly available communication among independently written applications. Cloud Pub/Sub delivers low-latency, durable messaging that helps developers quickly integrate systems hosted on the Google Cloud Platform and externally.

Detailed documentation can be found at watermill.io and in the Getting Started Google Cloud section.

Credits goes to @maclav3. Thanks for the contribution!

NATS Streaming Pub/Sub

NATS Streaming is a data streaming system powered by NATS, and written in the Go programming language. The executable name for the NATS Streaming server is nats-streaming-server. NATS Streaming embeds, extends, and interoperates seamlessly with the core NATS platform.

Detailed documentation can be found at watermill.io and in Getting Started NATS section.

Built in cooperation with @Nykakin, @maclav3 and @m110.

What’s next?

If there’s something missing in Watermill, feel free to post a new feature request or give thumbs up for the existing issue.

For now, Event sourcing, Sagas support and MySQL Binlog subscriber seem to be the most popular.

Also, the HTTP Publisher sounds like not too much of work and may provide some interesting functionalities.

Support

If you didn’t find an answer to your question in the documentation, you can hit me at Twitter: @roblaszczak.

Also, check out the #watermill channel on gophers.slack.com (invite here).