Benchmark Speed Test Method

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Benchmark Speed Test Method

4D Tech mailing list
We're getting a new Windows 2016 server for our 4DServer, and before I migrate, I'd like to write a benchmark speed test to run before and after. I'd like to measure create, edit, delete records, processing, IO, file copying, and other relevant functions. We're also migrating from v16.6 to v17 later in the year, and I'd like to run it again before and after the upgrade. I'm wondering if anyone has a method they've written for this purpose that they might be willing to share?

**********************************************************************
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

Re: Benchmark Speed Test Method

4D Tech mailing list
On Fri, Apr 5, 2019 at 3:00 PM  Neal Schaefer wrote:

> [snip]
>
> * We're getting a new Windows 2016 server for our 4DServer, and before I
> migrate, I'd like to write a benchmark speed test to run before and after.
> I'd like to measure create, edit, delete records, processing, IO, file
> copying, and other relevant functions. We're also migrating from v16.6 to
> v17 later in the year, and I'd like to run it again before and after the
> upgrade. I'm wondering if anyone has a method they've written for this
> purpose that they might be willing to share?*
>
We'd be very interested in that too. And Neal, I hope you share your
results when finished.

Also, has anyone moved up to v17 web server yet application? (Not "web
area", but rather the full "web server application") Can you tell how it
compares to v15 please? Any issues one should be aware of?
-------------------------
Stephen Simpson
Cimarron Software
**********************************************************************
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

Re: Benchmark Speed Test Method

4D Tech mailing list
You might want to check with the EULA on that. It’s been some years since I
checked but it did disallow publication of the results of performance
tests.


--
Douglas von Roeder
949-336-2902


On Fri, Apr 5, 2019 at 12:32 PM steve simpson via 4D_Tech <
[hidden email]> wrote:

> On Fri, Apr 5, 2019 at 3:00 PM  Neal Schaefer wrote:
>
> > [snip]
> >
> > * We're getting a new Windows 2016 server for our 4DServer, and before I
> > migrate, I'd like to write a benchmark speed test to run before and
> after.
> > I'd like to measure create, edit, delete records, processing, IO, file
> > copying, and other relevant functions. We're also migrating from v16.6 to
> > v17 later in the year, and I'd like to run it again before and after the
> > upgrade. I'm wondering if anyone has a method they've written for this
> > purpose that they might be willing to share?*
> >
> We'd be very interested in that too. And Neal, I hope you share your
> results when finished.
>
> Also, has anyone moved up to v17 web server yet application? (Not "web
> area", but rather the full "web server application") Can you tell how it
> compares to v15 please? Any issues one should be aware of?
> -------------------------
> Stephen Simpson
> Cimarron Software
> **********************************************************************
> 4D Internet Users Group (4D iNUG)
> Archive:  http://lists.4d.com/archives.html
> Options: https://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:[hidden email]
> **********************************************************************
**********************************************************************
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

RE: Benchmark Speed Test Method

4D Tech mailing list
Stephen,

We benchmark every Clients server storage system as part of our initial project management so we know if it will meet our requirements.  We use a Microsoft tool called DiskSpd and have a configuration script to test various loads on the storage system.  The results are plotted against one of our "base" systems for comparison, with the results provided to our Customer.  Our primary metrics are:

     1. IOPS
     2. MBPS
     3. Avg Latency
     4. Max Latency.

Most of our customers configure their storage system wrong, basically as a "data warehouse" not as a "OLTP" where most packet sizes will be 2 - 4 kb.  The latency of a storage system is very important, similar to how important it is for 4D's network connection.

Best,


Steve

*********************************************
  Stephen J. Orth                                                
  The Aquila Group, Inc.          Office:  (608) 834-9213
  P.O. Box 690                           Mobile:  (608) 347-6447
  Sun Prairie, WI 53590

  E-Mail:  [hidden email]
*********************************************

-----Original Message-----
From: 4D_Tech <[hidden email]> On Behalf Of Douglas von Roeder via 4D_Tech
Sent: Friday, April 5, 2019 2:39 PM
To: 4D iNug Technical <[hidden email]>
Cc: Douglas von Roeder <[hidden email]>
Subject: Re: Benchmark Speed Test Method

You might want to check with the EULA on that. It’s been some years since I checked but it did disallow publication of the results of performance tests.


--
Douglas von Roeder
949-336-2902


On Fri, Apr 5, 2019 at 12:32 PM steve simpson via 4D_Tech < [hidden email]> wrote:

> On Fri, Apr 5, 2019 at 3:00 PM  Neal Schaefer wrote:
>
> > [snip]
> >
> > * We're getting a new Windows 2016 server for our 4DServer, and
> > before I migrate, I'd like to write a benchmark speed test to run
> > before and
> after.
> > I'd like to measure create, edit, delete records, processing, IO,
> > file copying, and other relevant functions. We're also migrating
> > from v16.6 to
> > v17 later in the year, and I'd like to run it again before and after
> > the upgrade. I'm wondering if anyone has a method they've written
> > for this purpose that they might be willing to share?*
> >
> We'd be very interested in that too. And Neal, I hope you share your
> results when finished.
>
> Also, has anyone moved up to v17 web server yet application? (Not "web
> area", but rather the full "web server application") Can you tell how
> it compares to v15 please? Any issues one should be aware of?
> -------------------------
> Stephen Simpson
> Cimarron Software
> **********************************************************************
> 4D Internet Users Group (4D iNUG)
> Archive:  http://lists.4d.com/archives.html
> Options: https://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:[hidden email]
> **********************************************************************
**********************************************************************
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************

**********************************************************************
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

Re: Benchmark Speed Test Method

4D Tech mailing list
Steve:

Thanks for that information. I used to have one I had written in 4D. This tool will be superior I am certain of that. I will need to read up on this, as I have an install coming up.

Jody

> On Apr 5, 2019, at 2:04 PM, Stephen J. Orth via 4D_Tech <[hidden email]> wrote:
>
> Stephen,
>
> We benchmark every Clients server storage system as part of our initial project management so we know if it will meet our requirements.  We use a Microsoft tool called DiskSpd and have a configuration script to test various loads on the storage system.  The results are plotted against one of our "base" systems for comparison, with the results provided to our Customer.  Our primary metrics are:
>
>     1. IOPS
>     2. MBPS
>     3. Avg Latency
>     4. Max Latency.
>
> Most of our customers configure their storage system wrong, basically as a "data warehouse" not as a "OLTP" where most packet sizes will be 2 - 4 kb.  The latency of a storage system is very important, similar to how important it is for 4D's network connection.
>
> Best,
>
>
> Steve

**********************************************************************
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

RE: Benchmark Speed Test Method

4D Tech mailing list
Jody,

The reason we chose to go this route is so we can assure our customers of the results.  There is no bias in how the numbers were attained, or that they were possibly influenced by us.  Additionally, the ability to say this is a Microsoft tool really helps politically.

Most of our installation today are on VM sessions, and people don't know how to properly configure VM, or I should say the storage system being utilized by the VM.  This analysis will help indicate if they did not configure it properly.

Another reason we document this is for the day (and it will come) when they complain about system speed, the system is slow.  We simply run another analysis and compare it to the original, pre-installation numbers.  When it shows their hardware speed has changed, they have a much more difficult time blaming our software.   😉

I wish I didn't have to do this, but unfortunately it's more of a CYA issue.

Best,


Steve

*********************************************
  Stephen J. Orth                                                
  The Aquila Group, Inc.          Office:  (608) 834-9213
  P.O. Box 690                           Mobile:  (608) 347-6447
  Sun Prairie, WI 53590

  E-Mail:  [hidden email]
*********************************************

-----Original Message-----
From: Jody Bevan <[hidden email]>
Sent: Friday, April 5, 2019 4:03 PM
To: Stephen Orth <[hidden email]>; 4D iNug Technical <[hidden email]>
Subject: Re: Benchmark Speed Test Method

Steve:

Thanks for that information. I used to have one I had written in 4D. This tool will be superior I am certain of that. I will need to read up on this, as I have an install coming up.

Jody

> On Apr 5, 2019, at 2:04 PM, Stephen J. Orth via 4D_Tech <[hidden email]> wrote:
>
> Stephen,
>
> We benchmark every Clients server storage system as part of our initial project management so we know if it will meet our requirements.  We use a Microsoft tool called DiskSpd and have a configuration script to test various loads on the storage system.  The results are plotted against one of our "base" systems for comparison, with the results provided to our Customer.  Our primary metrics are:
>
>     1. IOPS
>     2. MBPS
>     3. Avg Latency
>     4. Max Latency.
>
> Most of our customers configure their storage system wrong, basically as a "data warehouse" not as a "OLTP" where most packet sizes will be 2 - 4 kb.  The latency of a storage system is very important, similar to how important it is for 4D's network connection.
>
> Best,
>
>
> Steve



**********************************************************************
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

RE: Benchmark Speed Test Method

4D Tech mailing list
In reply to this post by 4D Tech mailing list
> Also, has anyone moved up to v17 web server yet application? (Not "web
> area", but rather the full "web server application") Can you tell how it
> compares to v15 please? Any issues one should be aware of?

We are on V17, and didn't noticed any difference. But we didn't do a real benchmark.

Regards
Lutz

--  
Lutz Epperlein  
----------------------------------------------
Agendo Gesellschaft für politische Planung mbH
Köpenicker Str. 9
10997 Berlin
http://www.agendo.de/
----------------------------------------------


**********************************************************************
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

RE: Benchmark Speed Test Method

4D Tech mailing list
In reply to this post by 4D Tech mailing list
Steve,

In both v15.6 and v17.4 I had to put nginx in front of 4D Web Server to handle heavy loads.

Nginx did SSL termination, and it allows 4D to handle only the http processing.

So both 15.6 and 17.4 failed for me under very heavy loads, without a reverse-proxy (nginx) in front of them doing SSL termination.

David Ringsmuth

From: steve simpson via 4D_Tech
Sent: Friday, April 5, 2019 2:32 PM
To: [hidden email]
Cc: steve simpson
Subject: Re: Benchmark Speed Test Method

On Fri, Apr 5, 2019 at 3:00 PM  Neal Schaefer wrote:

> [snip]
>
> * We're getting a new Windows 2016 server for our 4DServer, and before I
> migrate, I'd like to write a benchmark speed test to run before and after.
> I'd like to measure create, edit, delete records, processing, IO, file
> copying, and other relevant functions. We're also migrating from v16.6 to
> v17 later in the year, and I'd like to run it again before and after the
> upgrade. I'm wondering if anyone has a method they've written for this
> purpose that they might be willing to share?*
>
We'd be very interested in that too. And Neal, I hope you share your
results when finished.

Also, has anyone moved up to v17 web server yet application? (Not "web
area", but rather the full "web server application") Can you tell how it
compares to v15 please? Any issues one should be aware of?
-------------------------
Stephen Simpson
Cimarron Software
**********************************************************************
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************

**********************************************************************
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

Re: Benchmark Speed Test Method

4D Tech mailing list
In reply to this post by 4D Tech mailing list
Steve,

Can you tell us what ranges you require/expect for each of those metrics?

Thanks,

Jim

On Fri, Apr 5, 2019 at 4:04 PM Stephen J. Orth via 4D_Tech <
[hidden email]> wrote:

> Stephen,
>
> We benchmark every Clients server storage system as part of our initial
> project management so we know if it will meet our requirements.  We use a
> Microsoft tool called DiskSpd and have a configuration script to test
> various loads on the storage system.  The results are plotted against one
> of our "base" systems for comparison, with the results provided to our
> Customer.  Our primary metrics are:
>
>      1. IOPS
>      2. MBPS
>      3. Avg Latency
>      4. Max Latency.
>
> Most of our customers configure their storage system wrong, basically as a
> "data warehouse" not as a "OLTP" where most packet sizes will be 2 - 4 kb.
> The latency of a storage system is very important, similar to how important
> it is for 4D's network connection.
>
> Best,
>
>
> Steve
>
> *********************************************
>   Stephen J. Orth
>   The Aquila Group, Inc.          Office:  (608) 834-9213
>   P.O. Box 690                           Mobile:  (608) 347-6447
>   Sun Prairie, WI 53590
>
>   E-Mail:  [hidden email]
> *********************************************
>
> -----Original Message-----
> From: 4D_Tech <[hidden email]> On Behalf Of Douglas von
> Roeder via 4D_Tech
> Sent: Friday, April 5, 2019 2:39 PM
> To: 4D iNug Technical <[hidden email]>
> Cc: Douglas von Roeder <[hidden email]>
> Subject: Re: Benchmark Speed Test Method
>
> You might want to check with the EULA on that. It’s been some years since
> I checked but it did disallow publication of the results of performance
> tests.
>
>
> --
> Douglas von Roeder
> 949-336-2902
>
>
> On Fri, Apr 5, 2019 at 12:32 PM steve simpson via 4D_Tech <
> [hidden email]> wrote:
>
> > On Fri, Apr 5, 2019 at 3:00 PM  Neal Schaefer wrote:
> >
> > > [snip]
> > >
> > > * We're getting a new Windows 2016 server for our 4DServer, and
> > > before I migrate, I'd like to write a benchmark speed test to run
> > > before and
> > after.
> > > I'd like to measure create, edit, delete records, processing, IO,
> > > file copying, and other relevant functions. We're also migrating
> > > from v16.6 to
> > > v17 later in the year, and I'd like to run it again before and after
> > > the upgrade. I'm wondering if anyone has a method they've written
> > > for this purpose that they might be willing to share?*
> > >
> > We'd be very interested in that too. And Neal, I hope you share your
> > results when finished.
> >
> > Also, has anyone moved up to v17 web server yet application? (Not "web
> > area", but rather the full "web server application") Can you tell how
> > it compares to v15 please? Any issues one should be aware of?
> > -------------------------
> > Stephen Simpson
> > Cimarron Software
> > **********************************************************************
> > 4D Internet Users Group (4D iNUG)
> > Archive:  http://lists.4d.com/archives.html
> > Options: https://lists.4d.com/mailman/options/4d_tech
> > Unsub:  mailto:[hidden email]
> > **********************************************************************
> **********************************************************************
> 4D Internet Users Group (4D iNUG)
> Archive:  http://lists.4d.com/archives.html
> Options: https://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:[hidden email]
> **********************************************************************
>
> **********************************************************************
> 4D Internet Users Group (4D iNUG)
> Archive:  http://lists.4d.com/archives.html
> Options: https://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:[hidden email]
> **********************************************************************
**********************************************************************
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************