Performance and Reliability Report

OpenIMSDK stress and reliability testing

Terminology and scope

• OpenIMSDK: The complete open-source project umbrella, consisting of the client SDK (OpenIMClientSDK) and the backend server (OpenIMServer).
• ChatServer: The business extension server. Unless stated otherwise, the core performance benchmarks on this page evaluate OpenIMServer without ChatServer.
• Application administrator: Administrative accounts used for platform management API calls (such as issuing tokens and executing batch migrations).
• Application business backend: The host application backend connecting to OpenIMServer, handling custom business logic and integrating via REST APIs, webhooks, or SDK events.

Test methodology

OpenIMSDK provides embeddable instant messaging infrastructure rather than a pre-packaged consumer chat application. Because OpenIMClientSDK core logic is implemented in Go via openim-sdk-core, deploying thousands of physical mobile devices for repeatable testing is impractical. These benchmarks use automated test harnesses to simulate large fleets of concurrent online clients and continuous message traffic to quantify system capacity, delivery reliability, and end-to-end latency.

Two coordinated benchmark tools were used:

1. Test Program A: openim-sdk-core/integration_test (reliability, latency, and state consistency verification)
2. Runs openim-sdk-core instances to emulate full OpenIMClientSDK behavior, validating the complete message receipt, decryption, and local SQLite storage pipeline.
3. Test Program B: openim-sdk-core/msgtest (stress and throughput generation)
4. Focuses on concurrent logins and high-volume message publishing to generate realistic peak traffic loads.

Recommended workflow: Use msgtest to generate background connection and throughput load, and use integration_test to perform sampled statistical validation of delivery reliability and round-trip latency.

Metrics definitions

• Reliability: Message delivery rate, defined as the percentage of messages published by senders that are successfully received and acknowledged by recipients.
• Latency: End-to-end elapsed time from the moment Client A creates and dispatches a message until Client B successfully receives and persists it locally.

Test environment

• Server 1: Ubuntu 22.04.2 LTS, 16 vCPUs, 64 GB RAM, 150 GB HDD. Hosts OpenIMServer and infrastructure dependencies (MongoDB, Redis, Kafka, Etcd, MinIO) and runs Test Program B. Test Program A can also run from /dev/shm shared memory.
• Server 2: Ubuntu 18.04.5 LTS, 4 vCPUs, 8 GB RAM, 40 GB HDD. Runs Test Program A from /dev/shm shared memory.
• Service scaling: In open-im-server/start-config.yml, workers were scaled to openim-push: 8 and openim-msgtransfer: 8. Other microservices ran as single instances.

Test scenarios and results

Test 1: Baseline load with 200 users

Test Program A simulates 200 users (100 logging in immediately and 100 logging in gradually), exchanging messages across friend chats and small groups.

Command:

go run main.go -lgr 0.5 -imf -crg -ckgn -ckcon -sem -ckmsn -u 200 -su 10 -lg 2 -cg 2 -cgm 5 -sm 5 -gm 5 -reg

Deployment: Test Program A runs in shared memory /dev/shm on Server 1.

Test 2: 50,000 concurrent online users with small groups

• Test Program B simulates 50,000 active online users publishing random messages to generate background system load.
• Test Program A simulates 100 users to perform sampled statistical measurements of delivery reliability and latency.

Benchmark commands:

# 1. Register 100,000 user accounts (Program A)
go run main.go -reg -u 100000

# 2. Maintain 50,000 concurrent connections generating load (Program B)
go run main.go -s 49500 -e 99500 -c 100 -i 500 -rs 1000 -rr 1000

# 3. Sample delivery reliability and latency (Program A)
go run main.go -lgr 0.8 -imf -crg -ckgn -ckcon -sem -ckmsn -u 100 -su 3 -lg 0 -cg 4 -cgm 5 -sm 100 -gm 100 -msgitv 1500 -test

Deployment: Test Program A runs in /dev/shm on Server 2; Test Program B runs on Server 1. The full benchmark test duration is approximately 4 hours.

Server resource utilization (Test 2)

Concurrent online connections:

Message throughput (metric indicates inbound message count per minute):

CPU utilization breakdown:

Resident memory footprint:

*Note: Figures are approximate process measurements and exclude Docker bridge container packet routing overhead.*

Test 3: 50,000 concurrent users with 50,000-member supergroups

• Test Program B generates background load with 50,000 concurrent online users sending random messages.
• Test Program A simulates 20 users (16 logging in immediately) sending messages to friend chats and 10 large groups with 50,000 members each.

Benchmark commands:

# Register 100,000 users
go run main.go -reg -u 100000

# Maintain 50,000 online connections
go run main.go -o 50000 -s 49500 -e 99500 -c 100 -i 500 -rs 1000 -rr 1000

# Measure message integrity and latency in 50,000-member groups
go run main.go -lgr 0.8 -imf -crg -ckgn -ckcon -sem -ckmsn -u 20 -su 3 -lg 10 -cg 0 -cgm 5 -sm 0 -gm 10

Deployment: Test Program A runs in /dev/shm on Server 2; Test Program B runs on Server 1.

Benchmark analysis

Under the tested hardware and architecture profile, OpenIMServer reliably sustained 50,000 concurrent online WebSocket connections and multiple 50,000-member supergroups. Under continuous throughput of ~1,700 messages per second, measured delivery reliability reached 100%, average round-trip latency was below 1 second, and peak latency remained under 3 seconds.

These benchmark figures reflect specific releases, hardware profiles, deployment topologies, and workload patterns, and should not be interpreted as absolute capacity guarantees for all production deployments. Perform load testing in your target staging environment with representative message sizes, user traffic patterns, storage engines, and high-availability configurations before launch.

Recommended production specifications

For a baseline deployment supporting 100,000 registered users, a 10% daily concurrent online rate (~10,000 CCU), 50,000-member supergroups, and sustained throughput of 600 messages per second:

*Calculations assume an average message payload size of 2 KB. Actual requirements depend on payload content; plain-text chat packets typically average ~700 bytes. For configuration parameters, execution flags, and step-by-step instructions on running the benchmark suites, see Benchmark tools.*