|
First stress test: Transfering 6 millions of records from a Flat File, to CSV file. |
|
As in most of the Test, there are several cases using differents techniques, setting of variables, with the same objective, satisfy the requirement and improve the performance. Always the first case is the best, and the rest of cases, are jobs without tuning, or an incomplet set of techniques (but helped on the way).
From the first test we had optimal results, but try to find the point of equilibrium between the saturation of the resources and benefits.
In this first test of Jasper will clarify it look like a dejavu with the analysis of another tool. But not. Technology partnership with Talend results in new open source ETL product that extends JasperSoft's open source BI suite. (there is an article covering this topic).
As we are concerned, like we're working with Talend (which I dislike because it's really interesting and very good performance), but we found a difference in results between the different tests, and all they won by Jasper.
This premature victory, I relate more with the installer of Jasper (at least I could get), where it becomes more "portable" and "light" compared with the same virtual machine with the appropriate installation of Talend.
|
LINKS
CASE 1:[137 secs.] CASE 2:[141 secs.] CASE 3:[153 secs.] |
|
| Objective: | To measure elapsed time reading and writing 6 million rows, from Flat file, to .CSV file, working on local disk. |
| Rows: | 6.024.000 M |
| Columns: | 37 Columns |
| Resources: | Virtual machine with: 2 GB RAM, Jasper like main process over the virtual plataform. The resources used are anecdotal, today, Any production environment has enough processing power for current and future requirements. The objective here, is to build, to execute and to measure with the same environment (regardless of the limited resources) |
|
Structure: (Metadata) |
* flexibility in the management of metadata |
| Design & Run |
|
| Elapsed time (s) | 137 Secs. |
| Rows per sec (avg) | 43.745 rows/sec |
| How to Improve Perform |
- Adjust the parameters: Xms -Xmx (as shown in the figure above) |
| Objective: | To measure elapsed time reading and writing 6 million rows, from Flat file, to .CSV file, working on local disk. |
| Rows: | 6.024.000 M |
| Columns: | 37 Columns |
| Resources: | Virtual machine with: 2 GB RAM, Jasper like main process over the virtual plataform. The resources used are anecdotal, today, Any production environment has enough processing power for current and future requirements. The objective here, is to build, to execute and to measure with the same environment (regardless of the limited resources) |
| Structure |
|
| Design & Run |  |
| Elapsed time | 141 Secs. |
| Rows per sec (avg) | 42.559 rows/sec |
| How to improve Perform | - Adjust the parameters: Xms -Xmx (as shown in the figure above |
| Objective: | To measure elapsed time reading and writing 6 million rows, from Flat file, to .CSV file, working on local disk. |
| Rows: | 6.024.000 M |
| Columns: | 37 Columns |
| Resources: | Virtual machine with: 2 GB RAM, Jasper like main process over the virtual plataform. The resources used are anecdotal, today, Any production environment has enough processing power for current and future requirements. The objective here, is to build, to execute and to measure with the same environment (regardless of the limited resources) |
| Structure |
|
| Design & Run |  |
| Elapsed time | 153 Secs. |
| Rows per sec (avg) | 39.311 rows/sec |
| How to improve Perform | - Adjust the parameters: Xms -Xmx (as shown in the figure above) |
