We’re all experiencing the woes of home office recently. You wake up hopefully and optimistic — then run into “Can you hear me?”, “You are muted” and “Would you please reconnect again as your video is frozen?”
So before we start why the Load test is essential nowadays for WebRTC Apps, let’s see first what’s WebRTC and do we really need it or we are already daily using it.
According to Google, in their 2017 WebRTC Update event, there are Over 1.5 billion weekly audio/video minutes — 214 million minutes a day and that’s only on Chrome. Moreover, we are using the following Apps/ Platforms on a daily basis which are using WebRTC technology on top of it:
- Google Meet
- Slack
- Facebook Messenger
- Amazon Chime
- House Party
- GoToMeeting
With WebRTC, you can add real-time communication capabilities to your application that works on top of an open standard. It supports video, voice, and generic data to be sent between peers, allowing developers to build powerful voice- and video communication solutions. Check more about WebRTC and how it’s different than Websocket in Reference below.
Now — back to our main Objective to this Article, How we can measure the stability, reliability, and scalability for Apps with Chat, Audio, and Video Call features. So in the next lines, we are going to share some Ideas to Build a load Test mechanism for WebRTC using Jmeter and Locust with Python. In the first part, we will guide you through tips and tricks for using Jmeter in detail for Load Testing on WebRTC and Websocket Apps.
Installation and Prerequisites
Download the latest Binaries JMeter
Download JMeter Plugin Manager for installing required Plugins
Install required plugins:
- For Websocker Msgs: WebSocket Samplers by Peter Doornbosch
- For Listeners and Reports: jpgc plugin — Standard Set
- For passing Variables and Data between multiple Thread: Inter-Thread Communication
- For executing multiple requests in parallel: Parallel Controller & Sampler
Jmeter Scenario Recording (HTTP(s) and Websocket)
There are multiple ways for recording a scenario and import Requests to Jmeter. You can check some on my earlier article. However, there are very important steps and notes we should take care of after recording. Following are:
- Cleansing the requests to get only HTTP/S requests and ignore Js files or images.
- The Client implementation of Requests. Most probably for Java Apps needs Client Implementation of Java(Especially for POST requests). That’s doable by changing Client Implementation from Advanced Tab > Client Implementation > Java
- URL Encode need to be checked when Params of Request are not encoded in the URL
- If the Load Scenario needs to check Js/Media or files to be called from Server, We need to check “Retrieve All Embedded Resources”. That’s doable by changing Client Implementation from Advanced Tab > Mark “Retrieve All Embedded Resources”
- It’s always recommended to check on Follow Redirects for HTTP/S Request and Keep-alive connection for simulating Browser implementation for HTTP/S Requests
- Sometimes Content-Encoding need to modified to UTF-8/16 like in cases for multiple Language Support
- By default Jmeter and Blazemeter adding Think-time or Constant Timer between requests according to the time taken while recording. You can change the value from Timer to change the time that needs to be waited for Jmeter to execute that HTTP/S Request.
- By default Blazemeter adding HTTP Header Manager for each HTTP/S Request which contains Request Header like User-Agents, Referrer URL, Content-Type.
Websocket Msg Interpreting
You can follow some steps to get the WebSocket msgs which is transmitted between server and client or P2P while specific action is happening. Following are steps to check these WebSocket msgs:
- Open Chrome Dev Tools > Open Network Tab
- Select the “WS” tab
- Press CTRL+R
- Click on Displayed Connection of WS (according to the type of WebSocket protocol used)
- Open Messages:
- Write Msgs are in Green(↑)
- Read Msgs are in White (↓)
There are four types of WebSocket msgs that must be implemented:
- Connect to WS (Write WS Msg) — handshake for connecting client to the WebSocket server, which is the very first step for the handshake from client to Websocket. To simulate connect to WebSocket msg:
- Add New “Websocket Single Write Sampler”
- Select Setup new Connection
- Add Server URL which displayed in Chrome Request Url of WebSocket
- Add Body to the request which exists in Request Body
- Correlate and Parameterize body values as needed. Check part of correlation and parameterization in my earlier article
2. Read WS Msg — This is the response from the Websocket server to any write Msg. These Read Msgs are used to validate and check Server Response after connection established successfully, Notification, ReadMsgs, Broadcast Msgs,..etc. To simulate Read WS Msg, you just need to:
- Add New “Websocket Single Read Sampler”
- Mark “Use existing connection” which was established before using Connect to WS msg
3. Write WS Msg — This type of WS is used when the client needs to communicate with the server, like in cases of sending a chat on a channel or applying a heartbeat mechanism. To simulate Write WS Msg:
- Add New “Websocket Single Write Sampler”
- Mark “Use existing connection”
- Add correlated Request data that displayed in Chrome WS Messages.
4. Websocket Close — This one is used to close the current opening Connection to Websocket. To simulate it:
- Add New “Websocket Single Write Sampler”
- Mark “Use existing connection”
- Add the Msg Request Data for disconnect a client from WS. Which contains something like /disconnect in the Request body
Alternatively, Websocket Close Request would achieve closing the connection between server and client
General Configurations
Following Config Elements are needed to be added to the Test Plan:
- HTTP Cookie Manager — Marking “Clear Cookies each iteration” for clear cookies before running each iteration
- HTTP Cache Manager — Marking “Clear Cache each iteration” for clear cache before running each iteration
- Loop Controls — This used for Looping or repeating some Requests multiple times or for infinite until the Duration of the Test Thread reached. For example, we are using this to keep Users sending and Receiving Msgs after Login multiple times
- Response Assertion — This used for validating the specific Request contains or doesn’t contain a value in Response of Text, Header, Code, or Msg. For example, we used this for validating broken and Unknow_session from WebSocket and extract its UUID. So you can add a new Assertion:
- Apply To Main Sample only
- Field to Test: Text Response
- Pattern Matching Rules: Not
- Pattern to Test: error”:”402::unknown_session”,” successful”: false
- Custom Failure message: descriptive msg with ${__threadNum} to catch the failed thread number
Listeners and Reporting
Following are some listeners that need to be added for debugging, interpreting, and reporting results:
- View Results Tree — Which reports all request details to be viewed as text, HTML, HTML Source Formatted and the Response Details include Header, Body, and response codes
- Response Time Over Time — This reports in Graph all Response Time for the Requests. It can be filtered as well to get specific Response Time for an API
- Transaction per Second — which reports overall transactions that occurred to the server in a second. That might be tricky for Websocket Msg, as for Send one Msg it required around 10 WS Write/Read MSG, however, it represents real transaction for HTTP/S Requests
- Response Latencies over Time — which reports the Latency on the Network side for getting the server whole response including data transfer
- Response Code over Time — which reports by graph the response codes over the execution time
- Active Threads over Time — which reports by graph the Active number of Users over the execution time
- Connect Time over Time — which reports by graph the overall time a request elapsed for establishing a successful connection
- Hits per Second — which reports by graph the hits to the server including child samples per second
Conclusion
Nowadays we are in touch daily with WebRTC Apps which mandate Software Engineers, Test Engineer, Performance Engineers, and all stakeholders to keep attention to how those Apps are really running in a sustainable fashion, how many users can take in place simultaneously, and how fastly it responds with client needs without facing instability and to be scalable platforms for communication.
In the next part, we will discuss how we can interpret the results from Load Testing along with Monitoring servers by adding alerts in case of failures, errors and get into request inject by following Response Times for each component. Moreover, We will discuss an alternative way for Load Testing of WebRTC apps using Locust with Selenium Webdriver to simulate real Audio and Video Calls on Browsers. See you soon and Stay tuned.
