setting up mirror

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

setting up mirror

4D Tech mailing list
Hi All,

I am finalizing code to set up a mirror and then an emergency server.

I have two trains of thought, both of which use a shared folder to move the log file into
Option 1 Create a second text file after the journal has been copied and on the mirror, have a process that runs every x minutes looking for both that file ad the journal. When it finds both, it integrates journal and moves it to an integrated directory and then deletes the text file

2. After copy of journal, use an FN call in sql to launch an integrate process on the mirror.

Any ideas which people would use more and feel more sure with. We will NOT be using soap or HTTPS to do this


Thanks and regards


Chuck
------------------------------------------------------------------------------------------------
 Chuck Miller Voice: (617) 739-0306
 Informed Solutions, Inc. Fax: (617) 232-1064      
 mailto:cjmiller<AT SIGN>informed-solutions.com
 Brookline, MA 02446 USA Registered 4D Developer                
       Providers of 4D and Sybase connectivity
          http://www.informed-solutions.com 
------------------------------------------------------------------------------------------------
This message and any attached documents contain information which may be confidential, subject to privilege or exempt from disclosure under applicable law.  These materials are intended only for the use of the intended recipient. If you are not the intended recipient of this transmission, you are hereby notified that any distribution, disclosure, printing, copying, storage, modification or the taking of any action in reliance upon this transmission is strictly prohibited.  Delivery of this message to any person other than the intended recipient shall not compromise or waive such confidentiality, privilege or exemption from disclosure as to this communication.

**********************************************************************
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

Re: setting up mirror

4D Tech mailing list
Hi chuck,

I have been using option 1 for years now.
With 4D server version 13.x, I needed to intervene on a regular basis to fix errors with the log files.

With 4D server version 15.x, this is solid as a rock. No interventions in the last 6 months.

Regards,

Rudy Mortier
Two Way Communications bvba



> On 14 May 2018, at 21:08, Chuck Miller via 4D_Tech <[hidden email]> wrote:
>
> Hi All,
>
> I am finalizing code to set up a mirror and then an emergency server.
>
> I have two trains of thought, both of which use a shared folder to move the log file into
> Option 1 Create a second text file after the journal has been copied and on the mirror, have a process that runs every x minutes looking for both that file ad the journal. When it finds both, it integrates journal and moves it to an integrated directory and then deletes the text file
>
> 2. After copy of journal, use an FN call in sql to launch an integrate process on the mirror.
>
> Any ideas which people would use more and feel more sure with. We will NOT be using soap or HTTPS to do this
>
>
> Thanks and regards
>
>
> Chuck
> ------------------------------------------------------------------------------------------------
> Chuck Miller Voice: (617) 739-0306
> Informed Solutions, Inc. Fax: (617) 232-1064      
> mailto:cjmiller<AT SIGN>informed-solutions.com
> Brookline, MA 02446 USA Registered 4D Developer                
>       Providers of 4D and Sybase connectivity
>          http://www.informed-solutions.com 
> ------------------------------------------------------------------------------------------------
> This message and any attached documents contain information which may be confidential, subject to privilege or exempt from disclosure under applicable law.  These materials are intended only for the use of the intended recipient. If you are not the intended recipient of this transmission, you are hereby notified that any distribution, disclosure, printing, copying, storage, modification or the taking of any action in reliance upon this transmission is strictly prohibited.  Delivery of this message to any person other than the intended recipient shall not compromise or waive such confidentiality, privilege or exemption from disclosure as to this communication.
>
> **********************************************************************
> 4D Internet Users Group (4D iNUG)
> FAQ:  http://lists.4d.com/faqnug.html
> Archive:  http://lists.4d.com/archives.html
> Options: https://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:[hidden email]
> **********************************************************************

**********************************************************************
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

Re: setting up mirror

4D Tech mailing list
Thanks for your update. I assume you have a process on mirror that runs and
delays itself for x amount of time. What happens if you try and create a
second log but first has not been integrated. What do you do? Would you
mind sharing code. I know it should be pretty simple, but I always would
prefer not to re-invent the wheel. If so, can you email me privately.
Thanks again

What my plan is to get a directory listing and then sort by date created

Regards

Chuck

On Tue, May 15, 2018 at 2:01 PM, Two Way Communications via 4D_Tech <
[hidden email]> wrote:

> Hi chuck,
>
> I have been using option 1 for years now.
> With 4D server version 13.x, I needed to intervene on a regular basis to
> fix errors with the log files.
>
> With 4D server version 15.x, this is solid as a rock. No interventions in
> the last 6 months.
>
> Regards,
>
> Rudy Mortier
> Two Way Communications bvba
>
>
>
> > On 14 May 2018, at 21:08, Chuck Miller via 4D_Tech <[hidden email]>
> wrote:
> >
> > Hi All,
> >
> > I am finalizing code to set up a mirror and then an emergency server.
> >
> > I have two trains of thought, both of which use a shared folder to move
> the log file into
> > Option 1 Create a second text file after the journal has been copied and
> on the mirror, have a process that runs every x minutes looking for both
> that file ad the journal. When it finds both, it integrates journal and
> moves it to an integrated directory and then deletes the text file
> >
> > 2. After copy of journal, use an FN call in sql to launch an integrate
> process on the mirror.
> >
> > Any ideas which people would use more and feel more sure with. We will
> NOT be using soap or HTTPS to do this
> >
> >
> > Thanks and regards
> >
> >
> > Chuck
> > ------------------------------------------------------------
> ------------------------------------
> > Chuck Miller Voice: (617) 739-0306
> > Informed Solutions, Inc. Fax: (617) 232-1064
> > mailto:cjmiller<AT SIGN>informed-solutions.com
> > Brookline, MA 02446 USA Registered 4D Developer
> >       Providers of 4D and Sybase connectivity
> >          http://www.informed-solutions.com
> > ------------------------------------------------------------
> ------------------------------------
> > This message and any attached documents contain information which may be
> confidential, subject to privilege or exempt from disclosure under
> applicable law.  These materials are intended only for the use of the
> intended recipient. If you are not the intended recipient of this
> transmission, you are hereby notified that any distribution, disclosure,
> printing, copying, storage, modification or the taking of any action in
> reliance upon this transmission is strictly prohibited.  Delivery of this
> message to any person other than the intended recipient shall not
> compromise or waive such confidentiality, privilege or exemption from
> disclosure as to this communication.
> >
> > **********************************************************************
> > 4D Internet Users Group (4D iNUG)
> > FAQ:  http://lists.4d.com/faqnug.html
> > Archive:  http://lists.4d.com/archives.html
> > Options: https://lists.4d.com/mailman/options/4d_tech
> > Unsub:  mailto:[hidden email]
> > **********************************************************************
>
> **********************************************************************
> 4D Internet Users Group (4D iNUG)
> FAQ:  http://lists.4d.com/faqnug.html
> Archive:  http://lists.4d.com/archives.html
> Options: https://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:[hidden email]
> **********************************************************************
>



--
-----------------------------------------------------------------------------------------
 Chuck Miller Voice: (617) 739-0306 Fax: (617) 232-1064
 Informed Solutions, Inc.
 Brookline, MA 02446 USA Registered 4D Developer
       Providers of 4D, Sybase & SQL Server connectivity
          http://www.informed-solutions.com
-----------------------------------------------------------------------------------------
This message and any attached documents contain information which may be
confidential, subject to privilege or exempt from disclosure under
applicable law.  These materials are intended only for the use of the
intended recipient. If you are not the intended recipient of this
transmission, you are hereby notified that any distribution, disclosure,
printing, copying, storage, modification or the taking of any action in
reliance upon this transmission is strictly prohibited.  Delivery of this
message to any person other than the intended recipient shall not
compromise or waive such confidentiality, privilege or exemption
from disclosure as to this communication.
**********************************************************************
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

Re: setting up mirror

4D Tech mailing list
In reply to this post by 4D Tech mailing list
On May 15, 2018, at 2:00 PM, Charles Miller wrote:

> Thanks for your update. I assume you have a process on mirror that runs and
> delays itself for x amount of time. What happens if you try and create a
> second log but first has not been integrated. What do you do? Would you
> mind sharing code. I know it should be pretty simple, but I always would
> prefer not to re-invent the wheel. If so, can you email me privately.

Journal files that get created by the “New log file” command are automatically named with a unique name. The name has the sequence number in it. Example: Datafile[0004-0034].journal.

The first number between square brackets is related to the backup number. Each time you do a BACKUP the first number get’s incremented. The second number gets incremented with each call to “New log file”.

> What my plan is to get a directory listing and then sort by date created

It is not necessary to sort by create date. The journal files names are designed and formatted so that you can sort by file name and always get the oldest journal files first.

Create date can also be problematic because if you copy a group of files from one server to another. Sometimes the create date and modify dates of files get changed. Sometime not. I’ve not figured out exactly what triggers this sometimes change to file creation/modification dates when coping between machines.

No need to over complicate things. 4D designed the log file/mirroring system so that you can rely on journal file names for sequencing and to ensure uniqueness.

Tim

*****************************************
Tim Nevels
Innovative Solutions
785-749-3444
[hidden email]
*****************************************

**********************************************************************
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

Re: setting up mirror

4D Tech mailing list
OK I guess I knew that but what happens if you never do a backup (We will
not be backing up the master data file any longer. Are You?) to numbers in
second part, when the last one is Datafile[0004-9999].journal

Regards

Chuck

On Tue, May 15, 2018 at 3:25 PM, Tim Nevels via 4D_Tech <
[hidden email]> wrote:

> On May 15, 2018, at 2:00 PM, Charles Miller wrote:
>
> > Thanks for your update. I assume you have a process on mirror that runs
> and
> > delays itself for x amount of time. What happens if you try and create a
> > second log but first has not been integrated. What do you do? Would you
> > mind sharing code. I know it should be pretty simple, but I always would
> > prefer not to re-invent the wheel. If so, can you email me privately.
>
> Journal files that get created by the “New log file” command are
> automatically named with a unique name. The name has the sequence number in
> it. Example: Datafile[0004-0034].journal.
>
> The first number between square brackets is related to the backup number.
> Each time you do a BACKUP the first number get’s incremented. The second
> number gets incremented with each call to “New log file”.
>
> > What my plan is to get a directory listing and then sort by date created
>
> It is not necessary to sort by create date. The journal files names are
> designed and formatted so that you can sort by file name and always get the
> oldest journal files first.
>
> Create date can also be problematic because if you copy a group of files
> from one server to another. Sometimes the create date and modify dates of
> files get changed. Sometime not. I’ve not figured out exactly what triggers
> this sometimes change to file creation/modification dates when coping
> between machines.
>
> No need to over complicate things. 4D designed the log file/mirroring
> system so that you can rely on journal file names for sequencing and to
> ensure uniqueness.
>
> Tim
>
> *****************************************
> Tim Nevels
> Innovative Solutions
> 785-749-3444
> [hidden email]
> *****************************************
>
> **********************************************************************
> 4D Internet Users Group (4D iNUG)
> FAQ:  http://lists.4d.com/faqnug.html
> Archive:  http://lists.4d.com/archives.html
> Options: https://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:[hidden email]
> **********************************************************************
>



--
-----------------------------------------------------------------------------------------
 Chuck Miller Voice: (617) 739-0306 Fax: (617) 232-1064
 Informed Solutions, Inc.
 Brookline, MA 02446 USA Registered 4D Developer
       Providers of 4D, Sybase & SQL Server connectivity
          http://www.informed-solutions.com
-----------------------------------------------------------------------------------------
This message and any attached documents contain information which may be
confidential, subject to privilege or exempt from disclosure under
applicable law.  These materials are intended only for the use of the
intended recipient. If you are not the intended recipient of this
transmission, you are hereby notified that any distribution, disclosure,
printing, copying, storage, modification or the taking of any action in
reliance upon this transmission is strictly prohibited.  Delivery of this
message to any person other than the intended recipient shall not
compromise or waive such confidentiality, privilege or exemption
from disclosure as to this communication.
**********************************************************************
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

Re: setting up mirror

4D Tech mailing list
It goes to 5 digits.

--
Jeffrey Kain
[hidden email]


> On May 15, 2018, at 3:35 PM, Charles Miller via 4D_Tech <[hidden email]> wrote:
>
> OK I guess I knew that but what happens if you never do a backup (We will
> not be backing up the master data file any longer. Are You?) to numbers in
> second part, when the last one is Datafile[0004-9999].journal

**********************************************************************
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

Re: setting up mirror

4D Tech mailing list
In reply to this post by 4D Tech mailing list


>Thanks for your update. I assume you have a process on mirror that runs and delays

>itself for x amount of time. What happens if you try and create a second log but

>first has not been integrated. What do you do? Would you mind sharing code.

>I know it should be pretty simple, but I always would prefer not to re-invent the

>wheel. If so, can you email me privately.

We serialize the segmented log files. That decouples the segmentation flow from the integration flow. We can take the mirror offline for a period of time, then bring it back up and it automatically catches up. It also makes it easy to multiple, redundant mirrors. We have 3 in our Production system. It also allows us to rebuild a data file from a specific starting point from logs.

Tom Benedict
Optum Inc

This e-mail, including attachments, may include confidential and/or
proprietary information, and may be used only by the person or entity
to which it is addressed. If the reader of this e-mail is not the intended
recipient or his or her authorized agent, the reader is hereby notified
that any dissemination, distribution or copying of this e-mail is
prohibited. If you have received this e-mail in error, please notify the
sender by replying to this message and delete this e-mail immediately.
**********************************************************************
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

Re: setting up mirror

4D Tech mailing list
I assume that means you rename the .journal file to something like
DataFileXXXXXXXXXX.journal, where X is nex number in line

or DataFile.XXXXXXXXXXXXX where extension is next number in line

or better yet DataFileYYYYMMDDHHMMSS.journal. That way unless run more than
once per second you will always get a unique name.

Also  when creating second text file to show that copy is complete. Are you
creating one file for each journal that is there so gthat if for some
reason, the integrate does not happen More than one file can be handled at
a time


On Tue, May 15, 2018 at 5:08 PM, Benedict, Tom <[hidden email]>
wrote:

>
>
> >Thanks for your update. I assume you have a process on mirror that runs
> and delays
>
> >itself for x amount of time. What happens if you try and create a second
> log but
>
> >first has not been integrated. What do you do? Would you mind sharing
> code.
>
> >I know it should be pretty simple, but I always would prefer not to
> re-invent the
>
> >wheel. If so, can you email me privately.
>
>
>
> We serialize the segmented log files. That decouples the segmentation flow
> from the integration flow. We can take the mirror offline for a period of
> time, then bring it back up and it automatically catches up. It also makes
> it easy to multiple, redundant mirrors. We have 3 in our Production system.
> It also allows us to rebuild a data file from a specific starting point
> from logs.
>
>
>
> Tom Benedict
>
> Optum Inc
>
>
>
>
> This e-mail, including attachments, may include confidential and/or
> proprietary information, and may be used only by the person or entity
> to which it is addressed. If the reader of this e-mail is not the intended
> recipient or his or her authorized agent, the reader is hereby notified
> that any dissemination, distribution or copying of this e-mail is
> prohibited. If you have received this e-mail in error, please notify the
> sender by replying to this message and delete this e-mail immediately.
>



--
-----------------------------------------------------------------------------------------
 Chuck Miller Voice: (617) 739-0306 Fax: (617) 232-1064
 Informed Solutions, Inc.
 Brookline, MA 02446 USA Registered 4D Developer
       Providers of 4D, Sybase & SQL Server connectivity
          http://www.informed-solutions.com
-----------------------------------------------------------------------------------------
This message and any attached documents contain information which may be
confidential, subject to privilege or exempt from disclosure under
applicable law.  These materials are intended only for the use of the
intended recipient. If you are not the intended recipient of this
transmission, you are hereby notified that any distribution, disclosure,
printing, copying, storage, modification or the taking of any action in
reliance upon this transmission is strictly prohibited.  Delivery of this
message to any person other than the intended recipient shall not
compromise or waive such confidentiality, privilege or exemption
from disclosure as to this communication.
**********************************************************************
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

Re: setting up mirror

4D Tech mailing list
I do not backup the master database.

The incremental log files are indeed numbered Datafile[0004-0034].journal, so no need in changing their names.

What I do, is set up the mirror to do the automatic backup, but without a log file. Whenever there is an issue with the master database, you can restore the backup of the mirror database and integrate all the incremental log files that have been generated after the last backup.

This setup is simple. but very effective.

Regards,

Rudy Mortier
Two Way Communications bvba



> On 15 May 2018, at 23:16, Charles Miller via 4D_Tech <[hidden email]> wrote:
>
> I assume that means you rename the .journal file to something like
> DataFileXXXXXXXXXX.journal, where X is nex number in line
>
> or DataFile.XXXXXXXXXXXXX where extension is next number in line
>
> or better yet DataFileYYYYMMDDHHMMSS.journal. That way unless run more than
> once per second you will always get a unique name.
>
> Also  when creating second text file to show that copy is complete. Are you
> creating one file for each journal that is there so gthat if for some
> reason, the integrate does not happen More than one file can be handled at
> a time
>
>
> On Tue, May 15, 2018 at 5:08 PM, Benedict, Tom <[hidden email]>
> wrote:
>
>>
>>
>>> Thanks for your update. I assume you have a process on mirror that runs
>> and delays
>>
>>> itself for x amount of time. What happens if you try and create a second
>> log but
>>
>>> first has not been integrated. What do you do? Would you mind sharing
>> code.
>>
>>> I know it should be pretty simple, but I always would prefer not to
>> re-invent the
>>
>>> wheel. If so, can you email me privately.
>>
>>
>>
>> We serialize the segmented log files. That decouples the segmentation flow
>> from the integration flow. We can take the mirror offline for a period of
>> time, then bring it back up and it automatically catches up. It also makes
>> it easy to multiple, redundant mirrors. We have 3 in our Production system.
>> It also allows us to rebuild a data file from a specific starting point
>> from logs.
>>
>>
>>
>> Tom Benedict
>>
>> Optum Inc
>>
>>
>>
>>
>> This e-mail, including attachments, may include confidential and/or
>> proprietary information, and may be used only by the person or entity
>> to which it is addressed. If the reader of this e-mail is not the intended
>> recipient or his or her authorized agent, the reader is hereby notified
>> that any dissemination, distribution or copying of this e-mail is
>> prohibited. If you have received this e-mail in error, please notify the
>> sender by replying to this message and delete this e-mail immediately.
>>
>
>
>
> --
> -----------------------------------------------------------------------------------------
> Chuck Miller Voice: (617) 739-0306 Fax: (617) 232-1064
> Informed Solutions, Inc.
> Brookline, MA 02446 USA Registered 4D Developer
>       Providers of 4D, Sybase & SQL Server connectivity
>          http://www.informed-solutions.com
> -----------------------------------------------------------------------------------------
> This message and any attached documents contain information which may be
> confidential, subject to privilege or exempt from disclosure under
> applicable law.  These materials are intended only for the use of the
> intended recipient. If you are not the intended recipient of this
> transmission, you are hereby notified that any distribution, disclosure,
> printing, copying, storage, modification or the taking of any action in
> reliance upon this transmission is strictly prohibited.  Delivery of this
> message to any person other than the intended recipient shall not
> compromise or waive such confidentiality, privilege or exemption
> from disclosure as to this communication.
> **********************************************************************
> 4D Internet Users Group (4D iNUG)
> FAQ:  http://lists.4d.com/faqnug.html
> Archive:  http://lists.4d.com/archives.html
> Options: https://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:[hidden email]
> **********************************************************************

**********************************************************************
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

RE: setting up mirror

4D Tech mailing list
Hi Rudy,

> What I do, is set up the mirror to do the automatic backup, but without a log file. Whenever there is an issue with the master database, you can restore the backup of the mirror database and integrate all the incremental log files that have been generated after the last backup.

If I understand this correctly, you backup the Mirror server (so the 4BK files are from the mirror).
Then, if a problem occurs, you restore the 4BK from the mirror server, and then integrate the Journal files from the primary server.

I don't think that would work; I am pretty sure you would get an error about the journal file being too recent/old or in some other way not corresponding to the backup...

I believe, the correct approach, is to move the remaining journal files to the mirror and let the mirror integrate them as it would normally do so (using the INTEGRATE MIRROR LOG FILE command).
^ This gets your mirror up-to-date.

From this point, you could then copy the up-to-date files back to the primary server, then perform a backup and reseed the mirror with the new files.

I hope that is clear.

Kind Regards,

Tim PENNER



**********************************************************************
4D Internet Users Group (4D iNUG)
FAQ:  http://lists.4d.com/faqnug.html
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************