|
The Second stress test involves, Transfer 6 millions of records from a Flat File, to a table in MySQL. Version 2 |
|
In the previous versions, the time obtained was '47 minutes'. With the new architecture, and modifying the Array Size, the time dropped to less than 10 minutes.
Times:
Architecture:
-Environment: Infraestructure composed of 3 nodes
- 1)ESXi 5.0:
1.a) Physical Datastore 1:VM ETLClover (12GB RAM - 2 Cores * 2 Sockets)
1.b) Physical Datastore 2:VM Database ServerMySQL/Oracle (4GB RAM - 2 Cores * 2 Sockets)
-2)Monitor Performance: VM Monitor ESXi + SQL Server 2008 (with 4 GB RAM)
-3)Operator ETL: ESXi Client (with 3 GB RAM)
CASE 1: Array SIZE 100.000 Rows
| Objective: |
-To measure elapsed time reading and writing 6 million rows, from Flat file, to .CSV file. -Compare performances in the 2 environments. -Analyze use of the resources |
| ETL Tool | IBM Datastage 8.1 |
| Rows: | 6.024.000 M |
| Columns: | 37Columns |
|
Structure: (Metadata) |
|
| Design & Run |
|
| LOG |
Log:
|
| Elapsed time(s) | 589 Secs. vs 2832 sec. (V1) |
| Rows p/s(avg) |
VERSION 2: Performance improvement. |
|
How to Improve Performance |
-Adjust the parameters: - Array Size |
USE OF RESOURCES:VM IBM DATASTAGE 8.1
| TOTAL |
Important:MemorySwap: 0 CPU/Datastore:CPU Usage Mhz / Datastore usage between12:21-12:31 Menmory:After several executions, the memory consumption remains stable in7,1 GB
|
| CPU: |
CPU Monitoring, "Passive and Active state" in different executions.Last Execution:12:21-12:31
|
| Memory: |
Memory Monitoring:Last Execution:12:21-12:31
|
| Network |
Network Monitoring:Last Execution:12:21-12:31
|
| DataST |
|
