CRM 4.0 Performance Toolkit - How to use

Nov 6, 2012 at 1:44 PM

Hello.


I am one of those who managed to run this magic tool, written by Microsoft. And I want to share my experience.

Let's begin.

Installation:

  1. First of all i installed MS CRM Performance Toolkit 4.0 (at that I also has installed MS Visual Studio 2005 and SQL Express 2005) with all necessary setting as described in "Microsoft Dynamics CRM 4 Performance Toolkit.docx".
  2. The next step I created 200 users (crmusrt1, crmusrt2.., crmusrt200) in Active Directory and launched "RunPopulation.cmd". After that, there was no any errors in file 'UserPopulate_Errors_...'.3. I started the 'EMDBLoader.exe' and it has also been successfully completed.

Creating Tests:

I've created a load test which included 2 tests: CreateNewLead and CreateNewContact, and 10 users who must use this test without any think 
times.

In detail (Edit load patternt setting for a load test scenario):

  1. On the Scenario window I've selected "Do not use think times".
  2. Load Pattern - Constant Load: 10 users.
  3. Test Mis - CreateNewLead and CreateNewContact.
  4. Browser Mix - Internet Explorer 6.0.
  5. Network Mix - LAN.
  6. Counter Sets - (Default).
  7. Run Settings - Warm-up duration: 00:00:00, Run duration 00:20:00, Sampling rate 5 secods. Maximum error details: 100. Validation level - Low - invoke validation rules marked low.

Running Tests:

I ran the test that has created, as described above and recieved following results:

In Summary.

Reuests:

  1. Total Request: 5.815
  2. Requests/sec: 9.88
  3. Falled Requests: 3.00
  4. Cached Requests 0

Test Cases:

  1. Total Tests: 1,162 
  2. Tests/Sec: 1,97 
  3. Fslled Tests 3.00

Counters.

  1. User Load: Range: 10, Min 10, Max 10, Avg 10, Last 10.2. 
  2. Request/Sec: Range 100, Min 0.0, Max 43.6, Avg 9.9, Last 0.0.3. 
  3. Avg. Response Time: Range 100, Min 0.2, Max 72.1, Avg 1.0, Last -.4. 
  4. Total Transactions: Range 10000, Min 0, Max 2,324, Avg 1,191, Last 2,324.

The questions:

  1. On the Chart, counters "Request/Sec" and "Total Transactions" did not rise linearly. Counter "Request/Sec" rise on 00:00 sec to 00:45 sec. Next on the 00:45 to 1:05 fall to zero. Then it begins to grow only on 02:20 
    sec to 03:05 sec and fall to zero after 03:05 and so on. Counter "Total Transactions" behaves the same with "Request/Sec" but with one difference. After growth, it does not fall, but It 
    just does not change its value: rise on 00:00 sec to 00:45 sec, then it rise on 02:20 to 03:05 and so on. I think, that Counters "Request/Sec" and "Total Transactions" should not be so changed. They should match the settings described above. Is not 
    it?
  2. In Summary window parameter "Tests/Sec" must be 10, as was configured for 10 users. But It value is 1,97. What does It mean?

Thank you for any response.

Coordinator
Nov 6, 2012 at 5:46 PM

Hi,

The load pattern chosen was constant load, that means all tests will kick off at the same time and 10 tests will run at the same time. When the very first test starts, it tries to load all the data that the toolkit will need into memory of the process from the EntityManager DB(the DB that the toolkit uses to store all it's data). This process takes a while. As such it is suggested that you set warm up duration of around 2 minutes so that it runs one test case which kicks off the loading of data into memory during that time. Once the warm up time ends, the actual load pattern starts which in your case seems to be constant load, and should run smoother as all the data is already in memory of the toolkit. 

As to why there are waves of action and inaction, it might have to do with the threadpool settings you have. Please ensure that you run with enough thread pool.

As to your first point, one is requests/second which will show you the snapshot at any given point of time while the total transactions is a cumulative number and will be ever increasing. If you look at transactions/second it will be similar to the requests/second.

Tests/second is the total number of tests that completed in one second. It won't be 10 as it depends on the speed at which the server is able to complete the actions called by each test. If you had 10 simulated users and each test finishes in one second then your tests/second will be 10, in your case it seems when 10 simulated users are calling the tests, each test might be taking around 5 seconds which will give you your rough 1.97 tests/second.

Thanks,

Adi.

Mar 7, 2014 at 10:39 AM
Hi,


We have a requirement to test the CRM Dynamics with outlook integration and web browser. We have identified CRM performance toolkit to do the performance testing. We went through the blogs and all the CRM performance toolkit documents, but could not come up with a proper solutions. Kindly let us know the approach to test the performance of the CRM Dynamics application integrated with outlook and web browser​.

Regards,
Ashish