ASP.NET Core – MVC xUnit Integration Testing

Recently I wanted to test my new ASP.NET Core MVC app, so that I can be sure my startup.cs configuration is correct, and especially focusing on the JSON parsing of it.

I straight away run into a few basic stumbling blocks that might help a few people out there!

Test Discovery

The first stumbling block was trying to get the tests discovered at all.

I had to select a test runner and add this to the root of my project.json:

{
  "testRunner": "xunit",
  ....
}

I tried to initially use xunit and dotnet-test-xunit versions 2.1.0 which is the officially released version at the time of writing.

But as this does not support Microsoft.NETCore.App 1.0.1, after a bit off Bing’ing around, I figured out that I need to use "dotnet-test-xunit": "2.2.0-preview2-build1029"

Referencing the MVC Project

The second stumbling block was trying to simply reference the MVC project, in this case, “Taskily”. So I added the following line:

"dependencies": {
  .....
  "Taskily": "1.0.0*"
}

But this was giving me a ‚Äúruntimes‚ÄĚ is an unsupported framework error, which I couldn’t for the life of me figure out. In the end, it turned out to be that because the Taskily project (or at least it was called Taskily in the .sln) actually resided in a folder called WebApplication1 so my solution structure looked like this

root
|global.json
|---src
    |---WebApplication1  <-- renamed to Taskily in .sln
    |---Taskily.Tests    <-- new test project

The way I got this to compile is

  1. Close the .sln
  2. Rename WebApplicaiton1 folder to Taskily (same as project name in .sln)
  3. Re-Open .sln (Taskily won’t load as it is looking for src/WebApplication1/Taskily.xproj
  4. Remove the reference to the Taskily project from the .sln.
  5. Re-Add the Taskily project which should now be in src/Taskily/Taskily.xproj
  6. Compile

The error message was so not intuitive,and was bugging me for ages!

MVC Integration Test

Next, I needed to somehow integrate the startup configuration, which contains the JSON.Net serializer settings, etc. in my tests so I followed the Microsofts Integration Testing Article which allows you to create an in memory host of your MVC app with custom such as the following:

public class ApiTests
{
    private readonly HttpClient _client;

    public ApiTests()
    {
        var server = new TestServer(new WebHostBuilder()
                .UseStartup<Startup>()
        );

        _client = server.CreateClient();
    }

    [Fact]
    public async void Get_Returns2Values()
    {
        var response = await _client.GetAsync("https://localhost/api/Alexa/");

        var res = await response.Content.ReadAsStringAsync();

        res.Should().Be(@"[""value1"",""value2""]");
    }
}

In order for this to run, I had to add the following NuGet package dependency in order to be able to use the AspNetCore TestServer class.

"Microsoft.AspNetCore.TestHost": "1.1.0-preview1-final", 

UserSecrets

This allowed me to compile and run the test, but straight away the test failed as I was using UserSecrets, as specified in Microsoft’s Safe storage of app secrets during development article, and one of my Middleware was throwing an exception that the value cannot be null.

So I added a user secrets Id into the test project.json:

{
  ....
  "userSecretsId": "aspnet-WebApplication1-19b75b8b-b10e-4fea-94e8-17c9f955732e"
}

and also added the SecretManager tool into the project via project.json which allows me to manage the user secrets for the project via command line:

{
  .....
  "tools": {
    "Microsoft.Extensions.SecretManager.Tools": "1.0.0-preview2-final"
  },
  .....
}

I then set my 2 user secrets that one of my AspnetCore middleware’s needed by

  1. dotnet restore (Ctrl+Shift+K, Ctrl+Shift+R in VS on the project)
  2. Open Command Prompt in the test folder
  3. run the following command

    dotnet user-secrets set ClientID MySecretClientIdValue dotnet user-secrets set ClientIDPassword PasswordThatYouWillNeverGuess

Environment

Now that the UserSecrets were setup, my test was still failing with the same error (that the ClientID is null, which is retrieved from UserSecrets).

After a bit of digging around I noticed the following:

if (env.IsDevelopment())
{
    // For more details on using the user secret store see http://go.microsoft.com/fwlink/?LinkID=532709
    builder.AddUserSecrets();

    // This will push telemetry data through Application Insights pipeline faster, allowing you to view results immediately.
    builder.AddApplicationInsightsSettings(developerMode: true);
}

which meant, that ASP.NET Core will only use UserSecrets when the server is running in Development environment. I could change this but it’s probably like that for a reason! So instead decided to set the environment variable on the TestServer by using the WebHostBuilder like the following:

var server = new TestServer(new WebHostBuilder()
        .UseEnvironment(EnvironmentName.Development) //<-- Setting Environment Variable
        .UseStartup<Startup>()
);

Now my test was running successfully and Bobs your uncle! ūüėÄ

Happy Coding

TDD – Mock.Throw Interface Call Verification Technique

First of all I did not know what to call this post, so it is what it is. If you have any better suggestions leave a comment!

Often I find myself writing a class which ties multiple classes together. I like to call these controllers. Therefore using a MVVMC (Model-View-ViewModel-Controller) pattern which is a combination of MVVM and MVC as I believe they suit each other very well.

One problem that bugs when I am doing this using a TDD approach is that my interface call verification methods fail because of either NotImplementedException or NullReferenceException because as I test the first interface call, the rest is not yet implemented, or the mocks return null.

Rant

For the TDD purists out there, no I do not write a non compilable test first. I develop using the following cycle:

  1. Think about what I need and how I would test it
  2. Create an empty class to represent what I need
  3. Create an empty method
  4. Add comments of what this method should do
  5. Change the comments to empty implementations (often another method which throws new NotImplementedException)
  6. Write a test for the first fake implementation
  7. Code the implementation
  8. Repeat 6-7 until no empty implementations left.
  9. Repeat steps 1-8 for each interface I used.

I guess you can call it Test Driven Design, rather than Development. Maybe one day it will¬†change, who knows, but currently that is how I enjoy coding the most, makes me most productive as well as churn out decent quality code. I guess that is an idea for another blog post ūüėÄ

Rant Over

The Technique

[Test]
[ExpectedException(typeof(MockCalled))]
public void ProcessData_CallsWorker()
{
    var list = new List {1};
 
    // Setup worker
    _worker.Setup(w => w.DoWork(list))
           .Throws(); // Technique
 
    // Act
    _controller.ProcessData(list);
}

 The Reason

The reason for adding throws is to stop the flow of the program as soon as it hits that¬†mock. I often use this¬†when I want to verify a call was made to a mock, and I don’t want to test the remaining logic or flow of the class under test.

 

Method Under Test

 

The above logic can be represented by the following code:

internal void ProcessData(List data)
{
    var newData = _worker.DoWork(data);
 
    if (newData.Count == 0)
        throw new NotImplementedException(); // Do Something else
 
    SaveData(newData);
}

Having such a method, my first test would be to make sure my worker gets called. The conventional way of testing this would be something along these lines (I am using Moq, but most Mocking frameworks are similar)

[Test]
public void ProcessData_CallsWorker()
{
    var list = new List {1};
 
    // Setup worker
    _worker.Setup(w => w.DoWork(list))
        .Verifiable();
 
    // Act
    _controller.ProcessData(list);
 
    _worker.VerifyAll();
}

But using this method, the code blows up with a NullReferenceException.

NullReference

Yes you could make the worker.DoWork(list) return a list but that would just make the test throw an NotImplementedException as neither of the other paths are implemented yet.

A better option is to make the mocked worker.DoWork(list) throw an exception:

_worker.Setup(w => w.DoWork(list))
    .Throws(new MockCalled());

You then let the MockCalled exception bubble up the call stack, and get handled by you inside your test:

try
{
    _controller.ProcessData(list); // Will throw when calls worker
}
catch (MockCalled)
{
}

You should throw a custom exception ( I have used MockCalled : Exception ) so that you know for a fact that when you receive a MockCalled exception, it could of only come from your mocked method.

If your testing framework allows for annotating a test with ExpectedType (NUnit and MSTest do, haven’t tried any others), then this becomes even cleaner:

[Test]
[ExpectedException(typeof(MockCalled))]
public void ProcessData_CallsWorker()
{
    var list = new List {1};
 
    // Setup worker
    _worker.Setup(w => w.DoWork(list))
        .Throws();
 
    // Act
    _controller.ProcessData(list);
}

In the above method, I wrapped the Throws(new MockCalled()) in an extension method .Throws()  for Moq as below:

public static void Throws(this ISetup mock)
    where TMock : class
{
    mock.Throws(new MockCalled());
}

 Or you could go even further and wrap the Setup call as well:

public static ISetup ThrowsOn(this Mock mock, Expression> expression) 
    where T : class
{
    return mock.Setup(expression).Throws();
}

 Giving you a slightly neater interface:

[Test]
[ExpectedException(typeof(MockCalled))]
public void ProcessData_CallsWorker()
{
    var list = new List {1};
 
    // Setup worker
    _worker.ThrowsOn(w => w.DoWork(list));
 
    // Act
    _controller.ProcessData(list);
}

 Other Usages

Imagine the following scenario

Same Class Usage

This represents a scenario where you want to test that another method on the class under test was called. The usual way of testing something like this would be to mark your ProcessData() method as public virtual, and then Mock the class under test verifying  that the ProcessData method was called.

public virtual List ProcessData(List data) { ... }
 
[Test]
public void Iteration_HasData_CallsProcessData()
{
    // Setup repo
    var list = new List {1};
    _repo.Setup(r => r.GetData()).Returns(list);
 
    _mock.Setup(x => x.ProcessData(list))
        .Verifiable();
 
    // Act
    _mock.Object.Iteration();
 
    // Assert
    _mock.VerifyAll();
}

There are 2 issues with that:

  1. You have to make the ProcessData method public virtual so that Moq can provide an implementation that registers the call.
  2. DataChanged logic will throw a NullReferenceException if it depends on the result of ProcessData, so you have to return a valid result from ProcessData.

The first point has never sat well with me, so this is where you can again use this technique.

The second point, well thats a pain each time, so rather than setting up a verifiable virtual implementation for ProcessData that return valid data for the method to progress gracefully why not make it throw the MockCalled exception?

[Test]
[ExpectedException(typeof(MockCalled))]
public void Iteration_HasData_CallsProcessDataThrow()
{
    // Setup repo
    var list = new List {1};
    _repo.Setup(r => r.GetData()).Returns(list);
    // Setup mock
    _mock.ThrowsOn(x => x.ProcessData(list));
        
    // Act
    _mock.Object.Iteration();
}

 This makes the test a bit cleaner, but there is still a flaw which does not sit well with me. That you have to mark the method as public virtual. This is probably something that I could take a look at and implement in Moq, but I will leave that for another day. At the moment what I have been doing which is working quite well, is making the first logical interface interaction throw the exception.

In the scenario above, the first call which happens right at the start of ProcessData is the IWorker.DoWork(), so I can make this method throw the exception, knowing that it should get to there gracefully at which point the exception will be thrown.

This allows the method to stay private or internal, and does not need to be virtual.

[Test]
[ExpectedException(typeof(MockCalled))]
public void Iteration_HasData_CallsProcessDataWhichCallsWorker()
{
    // Setup repo
    var list = new List {1};
    _repo.Setup(r => r.GetData()).Returns(list);
    // Setup Worker
    _worker.ThrowsOn(x => x.DoWork(list));
        
    // Act
    _controller.Iteration();
}

As always you can find the code @ GitHub on codePerf/20150319-TddMockThrowInterfaceCallVerificationTechnique

Rx Request Response Throttle

My most recent project at work was to introduce a Request Response messaging API into one our systems, as previously we were receiving a daily batch file with upwards of 4m entries when we only needed around 500-1,000 of them.

One of the restrictions¬†was that we had to have mechanisms in place to ensure we did not flood the Service API with 1,000’s of messages all in one go, as this would cause delays for all other systems using the same API, as this was a widely used and critical service.

We had to make sure we do not have more than 4 requests queued at any one time (the API had 4 worker processes) so that everyone gets their fair share of the service. This made a great exercise for Rx ūüėÄ

The system could be represented via this diagram 

2015-03-07 13_47_38-RxRequestResponse-OverviewOne of my favourite things that Rx brought to the table, was this concept of virtual time using schedulers which you can learn about in Lee Campbells РIntro to Rx free on-line book under Part 4 РTesting Rx.

I started with an initial controller, which subscribes to an injected IObservable<Request> and calls the service straight away for each request. Responses are handled similarly, the controller subscribes to an IObservable<Response> on the IApiService using the injected IObserver<Response>

public class RequestResponseController
{
    public RequestResponseController(IRequester requester, IResponseHandler responseHandler, 
        IApiService service, ISchedulers schedulers)
    {
        requester
            .ObserveOn(schedulers.ThreadPool)
            .Subscribe(request => service.SendRequest(request));
            
        service.Responses
            .ObserveOn(schedulers.ThreadPool)
            .Subscribe(responseHandler);
    }
}

With a simple test to prove this

[Test]
public void Should_get_respponse_when_request_received()
{
    _requests.OnNext(new Request(1));
    
    _schedulers.ThreadPool.AdvanceBy(1); // Send Request
    _requestsReceived.Should().HaveCount(1);
    
    _serviceScheduler.AdvanceBy(1); // Process Request
    
    _schedulers.ThreadPool.AdvanceBy(1); // Process Response
    _responsesProcessed.Should().HaveCount(1);
}

Which passed with the following output:

Advancing ThreadPool by 1
Received Request: 1
  
Advancing ServiceScheduler by 1
Sending Response: 1
   
Advancing ThreadPool by 1
Processed Response: 1

I then went on to publish 5 requests from the requester, of which only 4 should be sent to the service before receiving a response.

[Test]
public void Should_send_max_4_requests_at_one_time()
{
    // Schedule 5 requests
    _requests.OnNext(new Request(1));
    _requests.OnNext(new Request(2));
    _requests.OnNext(new Request(3));
    _requests.OnNext(new Request(4));
    _requests.OnNext(new Request(5));
    
    _schedulers.ThreadPool.AdvanceBy(5); // Attempt to send all 5 requests
    _requestsReceived.Should().HaveCount(4); // Check if only 4 requests received by service
}

 This test failed as expected, and gave the following results:

Advancing ThreadPool by 5
Received Request: 1
Received Request: 2
Received Request: 3
Received Request: 4
Received Request: 5
Expected collection to contain 4 item(s), but found 5.

 In order to achieve the throttling, I added a new Subject

  field which acts as a throttle to the requester, so that only 4 requests are active at any time.

private readonly Subject _throttle = new Subject();

This could be achieve by using the  IObservable.Zip() extension, which brings 2 sequences together, and publishes at the rate of the slowest sequence. I also immediately published 4 objects into the  _throttle sequence so that the first 4 requests would immediately fire.

public RequestResponseController(...)
{
    // Service Subscription stayed the same
 
    requester
        .ObserveOn(schedulers.ThreadPool)
        .Zip(_throttle,(request, i) => request) // <---------Added Zip
        .Subscribe(request => service.SendRequest(request));
 
    for (int i = 0; i < 4; i++) // Init _throttle with 4 requests
        _throttle.OnNext(i);
}

 This made the test pass, and could be represented by the following marble diagram: 

Throttle  |1234----------------------------
Requester |----1-2-3-4-5-------------------
Result    |----1-2-3-4---------------------

 The missing piece of the puzzle was to ensure that queued requests would be sent out as soon as a response comes back. This could be represented by the following test:

[Test]
public void Should_send__5th_queued_request_after_receiving_1st_response()
{
    // Schedule 5 requests
    _requests.OnNext(new Request(1));
    _requests.OnNext(new Request(2));
    _requests.OnNext(new Request(3));
    _requests.OnNext(new Request(4));
    _requests.OnNext(new Request(5));
    _requests.OnNext(new Request(6));
 
    _schedulers.ThreadPool.AdvanceBy(5); // Attempt to send all 5 requests
 
    _serviceScheduler.AdvanceBy(1);
    _requestsReceived.Should().HaveCount(5); // Check that 5th request received by service
}

 Which failed and gave the following output:

Advancing ThreadPool by 5
Received Request: 1
Received Request: 2
Received Request: 3
Received Request: 4
 
Advancing ServiceScheduler by 1
Sending Response: 1
Expected collection to contain 5 item(s), but found 4.

 The implementation for this was easy, just subscribe to the  IApiService.Response  sequence, with an Action that pushes a value onto the _throttle sequence.

public RequestResponseController(...)
{
    service.Responses
        .Subscribe(r => _throttle.OnNext(0));
        
    // Rest of the Constructor
}

 And could be represented by the following marble diagram:

Throttle  |1234--------------0--0--0--0--0--0--0-
Requester |----1-2-3-4-5-6-7----------------89---
Result    |----1-2-3-4-------5--6--7--------89---
Response  |------------------1--2--3--4--5--6--7-

Stay tuned as I will be blogging about the Time-out -> Retry implementation for this controller soon. Where if we don't receive a response within 30 seconds, we retry, and to handle multiple responses for the same request (e.g. due to the retry) so that we do not process duplicate responses.

As always the code can be found on GitHub @ michal-ciechan/codePerf/20150308-RxRequestResponse/