Introduction to Google Protocol Buffer

1. Overview

In this article, we’ll be looking at the Google Protocol Buffer (protobuf) – a well-known language-agnostic binary data format. We can define a file with a protocol and next, using that protocol, we can generate code in languages like Java, C++, C#, Go, or Python.

This is an introductory article to the format itself; if you want to see how to use the format with a Spring web application, have a look at this article.

2. Defining Maven Dependencies

To use protocol buffers is Java, we need to add a Maven dependency to a protobuf-java:

<dependency>
    <groupId>com.google.protobuf</groupId>
    <artifactId>protobuf-java</artifactId>
    <version>${protobuf.version}</version>
</dependency>

<properties>
    <protobuf.version>3.2.0</protobuf.version>
</properties>

3. Defining a Protocol

Let’s start with an example. We can define a very simple protocol in a protobuf format:

message Person {
    required string name = 1;
}

This is a protocol of a simple message of Person type that has only one required field – name that has a string type.

Let’s look at the more complex example of defining a protocol. Let’s say that we need to store person details in a protobuf format:

package protobuf;

package protobuf;

option java_package = "com.baeldung.protobuf";
option java_outer_classname = "AddressBookProtos";

message Person {
    required string name = 1;
    required int32 id = 2;
    optional string email = 3;

    repeated string numbers = 4;
}

message AddressBook {
    repeated Person people = 1;
}

Our protocol consists of two types of data: a Person and an AddressBook. After generating the code (more on this in the later section), those classes will be the inner classes inside the AddressBookProtos class.

When we want to define a field that is required – meaning that creating an object without such field will cause an Exception, we need to use a required keyword.

Creating a field with the optional keyword means that this field doesn’t need to be set. The repeated keyword is an array type of variable size.

All fields are indexed – the field that is denoted with number 1 will be saved as a first field in a binary file. Field marked with 2 will be saved next and so on. That gives us better control over how fields are laid out in the memory.

4. Generating Java Code From a Protobuf File

Once we define a file, we can generate code from it.

Firstly, we need to install protobuf on our machine. Once we do this, we can generate code by executing a protoc command:

protoc -I=. --java_out=. addressbook.proto

The protoc command will generate Java output file from our addressbook.proto file. The -I option specifies a directory in which a proto file resides. The java-out specifies a directory where the generated class will be created.

Generated class will have setters, getters, constructors and builders for our defined messages. It will also have some util methods for saving protobuf files and deserializing them from binary format to Java class.

5. Creating an Instance of Protobuf Defined Messages

We can easily use a generated code to create Java instance of a Person class:

String email = "[email protected]";
int id = new Random().nextInt();
String name = "Michael Program";
String number = "01234567890";
AddressBookProtos.Person person =
  AddressBookProtos.Person.newBuilder()
    .setId(id)
    .setName(name)
    .setEmail(email)
    .addNumbers(number)
    .build();

assertEquals(person.getEmail(), email);
assertEquals(person.getId(), id);
assertEquals(person.getName(), name);
assertEquals(person.getNumbers(0), number);

We can create a fluent builder by using a newBuilder() method on the desired message type. After setting up all required fields, we can call a build() method to create an instance of a Person class.

6. Serializing and Deserializing Protobuf

Once we create an instance of our Person class, we want to save that on disc in a binary format that is compatible with a created protocol. Let’s say that we want to create an instance of the AddressBook class and add one person to that object.

Next, we want to save that file on disc – there is a writeTo() util method in auto-generated code that we can use:

AddressBookProtos.AddressBook addressBook 
  = AddressBookProtos.AddressBook.newBuilder().addPeople(person).build();
FileOutputStream fos = new FileOutputStream(filePath);
addressBook.writeTo(fos);

After executing that method, our object will be serialized to binary format and saved on disc. To load that data from a disc and deserialize it back to the AddressBook object we can use a mergeFrom() method:

AddressBookProtos.AddressBook deserialized
  = AddressBookProtos.AddressBook.newBuilder()
    .mergeFrom(new FileInputStream(filePath)).build();
 
assertEquals(deserialized.getPeople(0).getEmail(), email);
assertEquals(deserialized.getPeople(0).getId(), id);
assertEquals(deserialized.getPeople(0).getName(), name);
assertEquals(deserialized.getPeople(0).getNumbers(0), number);

7. Conclusion

In this quick article, we introduced a standard for describing and storing data in a binary format – Google Protocol Buffer.

We created a simple protocol, created Java instance that complies with defined protocol. Next, we saw how to serialize and deserialize objects using protobuf.