CLOSE DOCUMENT can be slow depending on file size

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

CLOSE DOCUMENT can be slow depending on file size

Tony Ringsmuth
This is an FYI, after I've been banging my had on the wall for about 8 hours
trying to figure out why a process of mine was slow.

SHORT STORY:  big documents take longer to close with CLOSE DOCUMENT.

If you have opened a document, and in my case I'm writing data to a text
document, the larger the document is, the longer it takes to CLOSE DOCUMENT.

On an SSD, in my current scenario, a 30MB file takes about 350ms to close.
That might not seem like a lot, but I'm in a tight loop, and it adds up
fast.


Opening, appending, or writing to the document were near instantaneous.

--
Tony Ringsmuth
Business Brothers Inc.
763-420-8686



**********************************************************************
4D v13 is available now - with more than 200 new features to make
your applications richer and faster
http://www.4d.com/products/new.html

4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

Re: CLOSE DOCUMENT can be slow depending on file size

Charles Miller
On Thu, Nov 14, 2013 at 4:08 PM, Tony Ringsmuth <[hidden email]> wrote:

> This is an FYI, after I've been banging my had on the wall for about 8
> hours
> trying to figure out why a process of mine was slow.
>
> SHORT STORY:  big documents take longer to close with CLOSE DOCUMENT.
>
> If you have opened a document, and in my case I'm writing data to a text
> document, the larger the document is, the longer it takes to CLOSE
> DOCUMENT.
>
> On an SSD, in my current scenario, a 30MB file takes about 350ms to close.
> That might not seem like a lot, but I'm in a tight loop, and it adds up
> fast.
>

Why not switch to using blob to doc. In case of append, do a doc to blob
add your stuff to the blob and then blob to doc

If new well I am sure you get the idea.

This is much faster than sending packets any way

Regards


Chuck


--
-----------------------------------------------------------------------------------------
 Chuck Miller Voice: (617) 739-0306 Fax: (617) 232-1064
 Informed Solutions, Inc.
 Brookline, MA 02446 USA Registered 4D Developer
       Providers of 4D, Sybase & SQL Sever connectivity
          http://www.informed-solutions.com


-----------------------------------------------------------------------------------------
This message and any attached documents contain information which may be
confidential, subject to privilege or exempt from disclosure under
applicable law.  These materials are intended only for the use of the
intended recipient. If you are not the intended recipient of this
transmission, you are hereby notified that any distribution, disclosure,
printing, copying, storage, modification or the taking of any action in
reliance upon this transmission is strictly prohibited.  Delivery of this
message to any person other than the intended recipient shall not
compromise or waive such confidentiality, privilege or exemption
from disclosure as to this communication.

**********************************************************************
4D v13 is available now - with more than 200 new features to make
your applications richer and faster
http://www.4d.com/products/new.html

4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

Re: CLOSE DOCUMENT can be slow depending on file size

Tony Ringsmuth
Chuck,

On 11/14/13 3:42 PM, "Chuck Miller" <[hidden email]> wrote:
> Why not switch to using blob to doc. In case of append, do a doc to blob
> add your stuff to the blob and then blob to doc
> If not well I am sure you get the idea.
> This is much faster than sending packets any way

In some cases my documents are hundreds of mb.  Also, these are log files
that I am only appending to.  So, I think Append to document is definitely
more appropriate in this case.

Regards,

--
Tony Ringsmuth
Business Brothers Inc.
763-420-8686



**********************************************************************
4D v13 is available now - with more than 200 new features to make
your applications richer and faster
http://www.4d.com/products/new.html

4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

Re: CLOSE DOCUMENT can be slow depending on file size

Arnaud de Montard
In reply to this post by Charles Miller

Le 14 nov. 2013 à 22:42, Charles Miller a écrit :

> Why not switch to using blob to doc. In case of append, do a doc to blob
> add your stuff to the blob and then blob to doc

Hi,
you can also append blobs to a document:
  doc:=Append document(path)
  SEND PACKET(doc;myBlob)


> This is much faster than sending packets any way

i'm surprised that rewriting the whole document is faster than just adding at the end. How did you see that?

--
Arnaud de Montard




**********************************************************************
4D v13 is available now - with more than 200 new features to make
your applications richer and faster
http://www.4d.com/products/new.html

4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

Re: CLOSE DOCUMENT can be slow depending on file size

Chuck Miller-2
In reply to this post by Tony Ringsmuth
I think you should test or better yet create a new doc and use a shell script to combine the two

Regards


Chuck
On Nov 14, 2013, at 4:51 PM, Tony Ringsmuth wrote:

> In some cases my documents are hundreds of mb.  Also, these are log files
> that I am only appending to.  So, I think Append to document is definitely
> more appropriate in this case.
>
> Regards,

------------------------------------------------------------------------------------------------
 Chuck Miller Voice: (617) 739-0306
 Informed Solutions, Inc. Fax: (617) 232-1064      
 PO Box 1720 mailto:cjmiller<AT SIGN>informed-solutions.com
 Brookline, MA 02446 USA Registered 4D Developer                
       Providers of 4D and Sybase connectivity
          http://www.informed-solutions.com 
------------------------------------------------------------------------------------------------
This message and any attached documents contain information which may be confidential, subject to privilege or exempt from disclosure under applicable law.  These materials are intended only for the use of the intended recipient. If you are not the intended recipient of this transmission, you are hereby notified that any distribution, disclosure, printing, copying, storage, modification or the taking of any action in reliance upon this transmission is strictly prohibited.  Delivery of this message to any person other than the intended recipient shall not compromise or waive such confidentiality, privilege or exemption from disclosure as to this communication.





**********************************************************************
4D v13 is available now - with more than 200 new features to make
your applications richer and faster
http://www.4d.com/products/new.html

4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

Re: CLOSE DOCUMENT can be slow depending on file size

Tony Ringsmuth
In reply to this post by Tony Ringsmuth
OK:  Now I'm baffled.

I've re-tested this in a new, fresh application, and I can't find any
slowness.

I don't have time to keep testing this, but let me just conclude by saying
that my tests are now in-conclusive:
One application found great slowness in CLOSE DOCUMENT with big files,
another found none.

Tony



**********************************************************************
4D v13 is available now - with more than 200 new features to make
your applications richer and faster
http://www.4d.com/products/new.html

4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

Re: CLOSE DOCUMENT can be slow depending on file size

Jeffrey Kain
Antivirus scanning upon the close of the file?

There’s a huge performance difference between different AV products.


On Nov 15, 2013, at 11:54 AM, Tony Ringsmuth <[hidden email]> wrote:

> OK:  Now I'm baffled.
>
> I've re-tested this in a new, fresh application, and I can't find any
> slowness.
>
> I don't have time to keep testing this, but let me just conclude by saying
> that my tests are now in-conclusive:
> One application found great slowness in CLOSE DOCUMENT with big files,
> another found none.

**********************************************************************
4D v13 is available now - with more than 200 new features to make
your applications richer and faster
http://www.4d.com/products/new.html

4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

Re: CLOSE DOCUMENT can be slow depending on file size

Tony Ringsmuth
> Antivirus scanning upon the close of the file?

I wondered about that too, but, my tests were all done on the same machine,
with no change in AV software.

FYI, I'm using Microsoft Essentials.

Regards,

--
Tony Ringsmuth
Business Brothers Inc.
763-420-8686


On 11/15/13 10:58 AM, "Jeffrey Kain" <[hidden email]> wrote:

> Antivirus scanning upon the close of the file?
>
> There¹s a huge performance difference between different AV products.
>
>
> On Nov 15, 2013, at 11:54 AM, Tony Ringsmuth <[hidden email]> wrote:
>
>> OK:  Now I'm baffled.
>>
>> I've re-tested this in a new, fresh application, and I can't find any
>> slowness.
>>
>> I don't have time to keep testing this, but let me just conclude by saying
>> that my tests are now in-conclusive:
>> One application found great slowness in CLOSE DOCUMENT with big files,
>> another found none.



**********************************************************************
4D v13 is available now - with more than 200 new features to make
your applications richer and faster
http://www.4d.com/products/new.html

4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

Replacing ITK with 4DIC

Jack des Bouillons
We have limited but important use of ITK in our system, which is now a dead
issue as we have some machines using the new Mac OS.

I want to try to replace ITK with the 4DIC methods.

Two questions:

When using 4DIC, how do you set the connection to be SSL when using TCP
Send?

There are several places where ITK calls are used that deal with IPCMsg...is
there something comparable in 4DIC?  It doesn't seem evident in looking at
the 4DIC Commands.

Any help will be appreciated.

Thanks.

Jack des Bouillons
Hobbs/Herder Advertising



**********************************************************************
4D v13 is available now - with more than 200 new features to make
your applications richer and faster
http://www.4d.com/products/new.html

4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

Re: Replacing ITK with 4DIC

Wayne Stewart
Jack,

No idea about the specifics but also Rob Laveaux's NTK.

http://www.pluggers.nl/products/ntk-plugin/

Regards,

Wayne


On 19 November 2013 09:13, Jack Des Bouillons
<[hidden email]> wrote:

> We have limited but important use of ITK in our system, which is now a dead
> issue as we have some machines using the new Mac OS.
>
> I want to try to replace ITK with the 4DIC methods.
>
> Two questions:
>
> When using 4DIC, how do you set the connection to be SSL when using TCP
> Send?
>
> There are several places where ITK calls are used that deal with IPCMsg...is
> there something comparable in 4DIC?  It doesn't seem evident in looking at
> the 4DIC Commands.
>
> Any help will be appreciated.
>
> Thanks.
>
> Jack des Bouillons
> Hobbs/Herder Advertising
>
>
>
> **********************************************************************
> 4D v13 is available now - with more than 200 new features to make
> your applications richer and faster
> http://www.4d.com/products/new.html
>
> 4D Internet Users Group (4D iNUG)
> FAQ:  http://lists.4d.com/faqnug.html
> Archive:  http://lists.4d.com/archives.html
> Options: https://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:[hidden email]
> **********************************************************************

**********************************************************************
4D v13 is available now - with more than 200 new features to make
your applications richer and faster
http://www.4d.com/products/new.html

4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

Re: Replacing ITK with 4DIC

Jack des Bouillons
Yes, I know about NTK...

Might be the way to go, but for what we use ITK for, I think I would try to
handle it with the TCP commands in 4DIC...

The messaging is a different proposition...I've been told that this can be
replaced by 4D code..I'm looking into that.

Thanks for the thoughts.

Jack des Bouillons

On 11/18/13 4:09 PM, "Wayne Stewart" <[hidden email]> wrote:

> Jack,
>
> No idea about the specifics but also Rob Laveaux's NTK.
>
> http://www.pluggers.nl/products/ntk-plugin/
>
> Regards,
>
> Wayne



**********************************************************************
4D v13 is available now - with more than 200 new features to make
your applications richer and faster
http://www.4d.com/products/new.html

4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

Re: Replacing ITK with 4DIC

Arnaud de Montard

Le 19 nov. 2013 à 01:19, Jack Des Bouillons a écrit :

> The messaging is a different proposition...I've been told that this can be
> replaced by 4D code..I'm looking into that.

<http://forums.4d.fr/Post/FR/8119810/1/8119811#8119811>

--
Arnaud de Montard




**********************************************************************
4D v13 is available now - with more than 200 new features to make
your applications richer and faster
http://www.4d.com/products/new.html

4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

Re: Replacing ITK with 4DIC

Benedict, Tom
In reply to this post by Jack des Bouillons
Hi Jack,

>When using 4DIC, how do you set the connection to be SSL when using TCP Send?
You use the sessionSettings parameter of TCP_Open. Set it to 2 for Synchronous SSL or 3 for Asynchronous SSL.

>There are several places where ITK calls are used that deal with IPCMsg...is there something comparable in 4DIC?
Sorry I can't help you on this one.

HTH,

Tom Benedict
Alere Wellbeing, inc

**********************************************************************
4D v13 is available now - with more than 200 new features to make
your applications richer and faster
http://www.4d.com/products/new.html

4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

RE: Replacing ITK with 4DIC

Witton, David
In reply to this post by Jack des Bouillons
Bear in mind there are a few gotchas in 4DIC that may bite you - I detailed some of them in a post a few weeks ago.

David Witton
Director, Information Systems
St. John's Medical Center
307.739.7383


-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Jack Des Bouillons
Sent: Monday, November 18, 2013 3:13 PM
To: 4D iNug Technical
Subject: Replacing ITK with 4DIC

We have limited but important use of ITK in our system, which is now a dead issue as we have some machines using the new Mac OS.

I want to try to replace ITK with the 4DIC methods.

Two questions:

When using 4DIC, how do you set the connection to be SSL when using TCP Send?

There are several places where ITK calls are used that deal with IPCMsg...is there something comparable in 4DIC?  It doesn't seem evident in looking at the 4DIC Commands.

Any help will be appreciated.

Thanks.

Jack des Bouillons
Hobbs/Herder Advertising



**********************************************************************
4D v13 is available now - with more than 200 new features to make your applications richer and faster http://www.4d.com/products/new.html

4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************

________________________________

Correspondence, including e-mail and other electronic communications, to and from employees and elected officials of the Teton County Hospital District, dba St. John's Medical Center, may be subject to disclosure under the Wyoming Public Records Act. This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender by reply e-mail and destroy all copies of the original message. Thank you for your cooperation.

**********************************************************************
4D v13 is available now - with more than 200 new features to make
your applications richer and faster
http://www.4d.com/products/new.html

4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************