REST Query Language – Advanced Search Operations

1. Overview

In this article, we’ll extend the REST Query Language we developed in the previous parts of the series to include more search operations.

We now support the following operations: Equality, Negation, Greater than, Less than, Starts with, Ends with, Contains and Like.

Note that we explored three implementations – JPA Criteria, Spring Data JPA Specifications and Query DSL; we’re going forward with Specifications in this article because it’s a clean and flexible way to represent our operations.

2. The SearchOperation enum

First – let’s start by defining a better representation of our various supported search operations – via an enumeration:

public enum SearchOperation {
    EQUALITY, NEGATION, GREATER_THAN, LESS_THAN, LIKE, STARTS_WITH, ENDS_WITH, CONTAINS;

    public static final String[] SIMPLE_OPERATION_SET = { ":", "!", ">", "<", "~" };

    public static SearchOperation getSimpleOperation(char input) {
        switch (input) {
        case ':':
            return EQUALITY;
        case '!':
            return NEGATION;
        case '>':
            return GREATER_THAN;
        case '<':
            return LESS_THAN;
        case '~':
            return LIKE;
        default:
            return null;
        }
    }
}

We have two sets of operations:

1. Simple – can be represented by one character:

• Equality: represented by colon (:)
• Negation: represented by Exclamation mark (!)
• Greater than: represented by (>)
• Less than: represented by (<)
• Like: represented by tilde (~)

2. Complex – need more than one character to be represented:

• Starts with: represented by (=prefix*)
• Ends with: represented by (=*suffix)
• Contains: represented by (=*substring*)

We also need to modify our SearchCriteria class to use the new SearchOperation:

public class SearchCriteria {
    private String key;
    private SearchOperation operation;
    private Object value;
}

3. Modify UserSpecification

Now – let’s include the newly supported operations into our UserSpecification implementation:

public class UserSpecification implements Specification<User> {

    private SearchCriteria criteria;

    @Override
    public Predicate toPredicate(
      Root<User> root, CriteriaQuery<?> query, CriteriaBuilder builder) {
    
        switch (criteria.getOperation()) {
        case EQUALITY:
            return builder.equal(root.get(criteria.getKey()), criteria.getValue());
        case NEGATION:
            return builder.notEqual(root.get(criteria.getKey()), criteria.getValue());
        case GREATER_THAN:
            return builder.greaterThan(root.<String> get(
              criteria.getKey()), criteria.getValue().toString());
        case LESS_THAN:
            return builder.lessThan(root.<String> get(
              criteria.getKey()), criteria.getValue().toString());
        case LIKE:
            return builder.like(root.<String> get(
              criteria.getKey()), criteria.getValue().toString());
        case STARTS_WITH:
            return builder.like(root.<String> get(criteria.getKey()), criteria.getValue() + "%");
        case ENDS_WITH:
            return builder.like(root.<String> get(criteria.getKey()), "%" + criteria.getValue());
        case CONTAINS:
            return builder.like(root.<String> get(
              criteria.getKey()), "%" + criteria.getValue() + "%");
        default:
            return null;
        }
    }
}

4. Persistence Tests

Next – we let’s test our new search operations – at the persistence level:

4.1. Test Equality

In the following example – we’ll search for a user by their first and last name:

@Test
public void givenFirstAndLastName_whenGettingListOfUsers_thenCorrect() {
    UserSpecification spec = new UserSpecification(
      new SearchCriteria("firstName", SearchOperation.EQUALITY, "john"));
    UserSpecification spec1 = new UserSpecification(
      new SearchCriteria("lastName", SearchOperation.EQUALITY, "doe"));
    List<User> results = repository.findAll(Specification.where(spec).and(spec1));

    assertThat(userJohn, isIn(results));
    assertThat(userTom, not(isIn(results)));
}

4.2. Test Negation

Next, let’s search for users that by the their first name not “john”:

@Test
public void givenFirstNameInverse_whenGettingListOfUsers_thenCorrect() {
    UserSpecification spec = new UserSpecification(
      new SearchCriteria("firstName", SearchOperation.NEGATION, "john"));
    List<User> results = repository.findAll(Specification.where(spec));

    assertThat(userTom, isIn(results));
    assertThat(userJohn, not(isIn(results)));
}

4.3. Test Greater Than

Next – we will search for users with age greater than “25”:

@Test
public void givenMinAge_whenGettingListOfUsers_thenCorrect() {
    UserSpecification spec = new UserSpecification(
      new SearchCriteria("age", SearchOperation.GREATER_THAN, "25"));
    List<User> results = repository.findAll(Specification.where(spec));

    assertThat(userTom, isIn(results));
    assertThat(userJohn, not(isIn(results)));
}

4.4. Test Starts With

Next – users with their first name starting with “jo”:

@Test
public void givenFirstNamePrefix_whenGettingListOfUsers_thenCorrect() {
    UserSpecification spec = new UserSpecification(
      new SearchCriteria("firstName", SearchOperation.STARTS_WITH, "jo"));
    List<User> results = repository.findAll(spec);

    assertThat(userJohn, isIn(results));
    assertThat(userTom, not(isIn(results)));
}

4.5. Test Ends With

Next we’ll search for users with their first name ending with “n”:

@Test
public void givenFirstNameSuffix_whenGettingListOfUsers_thenCorrect() {
    UserSpecification spec = new UserSpecification(
      new SearchCriteria("firstName", SearchOperation.ENDS_WITH, "n"));
    List<User> results = repository.findAll(spec);

    assertThat(userJohn, isIn(results));
    assertThat(userTom, not(isIn(results)));
}

4.6. Test Contains

Now, we’ll search for users with their first name containing “oh”:

@Test
public void givenFirstNameSubstring_whenGettingListOfUsers_thenCorrect() {
    UserSpecification spec = new UserSpecification(
      new SearchCriteria("firstName", SearchOperation.CONTAINS, "oh"));
    List<User> results = repository.findAll(spec);

    assertThat(userJohn, isIn(results));
    assertThat(userTom, not(isIn(results)));
}

4.7. Test Range

Finally, we’ll search for users with ages between “20” and “25”:

@Test
public void givenAgeRange_whenGettingListOfUsers_thenCorrect() {
    UserSpecification spec = new UserSpecification(
      new SearchCriteria("age", SearchOperation.GREATER_THAN, "20"));
    UserSpecification spec1 = new UserSpecification(
      new SearchCriteria("age", SearchOperation.LESS_THAN, "25"));
    List<User> results = repository.findAll(Specification.where(spec).and(spec1));

    assertThat(userJohn, isIn(results));
    assertThat(userTom, not(isIn(results)));
}

5. The UserSpecificationBuilder

Now that persistence is done and tested, let’s move our attention to the web layer.

We’ll build on top of the UserSpecificationBuilder implementation from the previous article to incorporate the new new search operations:

public class UserSpecificationsBuilder {

    private List<SearchCriteria> params;

    public UserSpecificationsBuilder with(
      String key, String operation, Object value, String prefix, String suffix) {
    
        SearchOperation op = SearchOperation.getSimpleOperation(operation.charAt(0));
        if (op != null) {
            if (op == SearchOperation.EQUALITY) {
                boolean startWithAsterisk = prefix.contains("*");
                boolean endWithAsterisk = suffix.contains("*");

                if (startWithAsterisk && endWithAsterisk) {
                    op = SearchOperation.CONTAINS;
                } else if (startWithAsterisk) {
                    op = SearchOperation.ENDS_WITH;
                } else if (endWithAsterisk) {
                    op = SearchOperation.STARTS_WITH;
                }
            }
            params.add(new SearchCriteria(key, op, value));
        }
        return this;
    }

    public Specification<User> build() {
        if (params.size() == 0) {
            return null;
        }

        Specification result = new UserSpecification(params.get(0));
     
        for (int i = 1; i < params.size(); i++) {
            result = params.get(i).isOrPredicate()
              ? Specification.where(result).or(new UserSpecification(params.get(i))) 
              : Specification.where(result).and(new UserSpecification(params.get(i)));
        }

        return result;
    }
}

6. The UserController

Next – we need to modify our UserController to correctly parse the new operations:

@RequestMapping(method = RequestMethod.GET, value = "/users")
@ResponseBody
public List<User> findAllBySpecification(@RequestParam(value = "search") String search) {
    UserSpecificationsBuilder builder = new UserSpecificationsBuilder();
    String operationSetExper = Joiner.on("|").join(SearchOperation.SIMPLE_OPERATION_SET);
    Pattern pattern = Pattern.compile(
      "(\\w+?)(" + operationSetExper + ")(\p{Punct}?)(\\w+?)(\p{Punct}?),");
    Matcher matcher = pattern.matcher(search + ",");
    while (matcher.find()) {
        builder.with(
          matcher.group(1), 
          matcher.group(2), 
          matcher.group(4), 
          matcher.group(3), 
          matcher.group(5));
    }

    Specification<User> spec = builder.build();
    return dao.findAll(spec);
}

We can now hit the API and get back the right results with any combination of criteria. For example – here’s a what a complex operation would look like using API with the query language:

http://localhost:8082/spring-rest-query-language/auth/users?search=firstName:jo*,age<25

And the response:

[{
    "id":1,
    "firstName":"john",
    "lastName":"doe",
    "email":"[email protected]",
    "age":24
}]

7. Tests for the Search API

Finally – let’s make sure our API works well by writing a suite of API tests.

We’ll start with the simple configuration of the test and the data initialization:

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(
  classes = { ConfigTest.class, PersistenceConfig.class }, 
  loader = AnnotationConfigContextLoader.class)
@ActiveProfiles("test")
public class JPASpecificationLiveTest {

    @Autowired
    private UserRepository repository;

    private User userJohn;
    private User userTom;

    private final String URL_PREFIX = "http://localhost:8082/spring-rest-query-language/auth/users?search=";

    @Before
    public void init() {
        userJohn = new User();
        userJohn.setFirstName("John");
        userJohn.setLastName("Doe");
        userJohn.setEmail("[email protected]");
        userJohn.setAge(22);
        repository.save(userJohn);

        userTom = new User();
        userTom.setFirstName("Tom");
        userTom.setLastName("Doe");
        userTom.setEmail("[email protected]");
        userTom.setAge(26);
        repository.save(userTom);
    }

    private RequestSpecification givenAuth() {
        return RestAssured.given().auth()
                                  .preemptive()
                                  .basic("username", "password");
    }
}

7.1. Test Equality

First – let’s search for a user with the first name “john” and last name “doe“:

@Test
public void givenFirstAndLastName_whenGettingListOfUsers_thenCorrect() {
    Response response = givenAuth().get(URL_PREFIX + "firstName:john,lastName:doe");
    String result = response.body().asString();

    assertTrue(result.contains(userJohn.getEmail()));
    assertFalse(result.contains(userTom.getEmail()));
}

7.2. Test Negation

Now – we’ll search for users when their first name isn’t “john”:

@Test
public void givenFirstNameInverse_whenGettingListOfUsers_thenCorrect() {
    Response response = givenAuth().get(URL_PREFIX + "firstName!john");
    String result = response.body().asString();

    assertTrue(result.contains(userTom.getEmail()));
    assertFalse(result.contains(userJohn.getEmail()));
}

7.3. Test Greater Than

Next – we will look for users with age greater than “25”:

@Test
public void givenMinAge_whenGettingListOfUsers_thenCorrect() {
    Response response = givenAuth().get(URL_PREFIX + "age>25");
    String result = response.body().asString();

    assertTrue(result.contains(userTom.getEmail()));
    assertFalse(result.contains(userJohn.getEmail()));
}

7.4. Test Starts With

Next – users with their first name starting with “jo”:

@Test
public void givenFirstNamePrefix_whenGettingListOfUsers_thenCorrect() {
    Response response = givenAuth().get(URL_PREFIX + "firstName:jo*");
    String result = response.body().asString();

    assertTrue(result.contains(userJohn.getEmail()));
    assertFalse(result.contains(userTom.getEmail()));
}

7.5. Test Ends With

Now – users with their first name ending with “n”:

@Test
public void givenFirstNameSuffix_whenGettingListOfUsers_thenCorrect() {
    Response response = givenAuth().get(URL_PREFIX + "firstName:*n");
    String result = response.body().asString();

    assertTrue(result.contains(userJohn.getEmail()));
    assertFalse(result.contains(userTom.getEmail()));
}

7.6. Test Contains

Next, we’ll search for users with their first name containing “oh”:

@Test
public void givenFirstNameSubstring_whenGettingListOfUsers_thenCorrect() {
    Response response = givenAuth().get(URL_PREFIX + "firstName:*oh*");
    String result = response.body().asString();

    assertTrue(result.contains(userJohn.getEmail()));
    assertFalse(result.contains(userTom.getEmail()));
}

7.7. Test Range

Finally, we’ll search for users with ages between “20” and “25”:

@Test
public void givenAgeRange_whenGettingListOfUsers_thenCorrect() {
    Response response = givenAuth().get(URL_PREFIX + "age>20,age<25");
    String result = response.body().asString();

    assertTrue(result.contains(userJohn.getEmail()));
    assertFalse(result.contains(userTom.getEmail()));
}

8. Conclusion

In this article we brought the query language of our REST Search API forward to a mature, tested, production-grade implementation. We now support a wide variety of operations and constraints, which should make it quite easy to cut across any dataset elegantly and get to the exact resources we’re looking for.