Shutdown and Memory Mapped Files

[Hugo <hugh<underbar>]

Hi

I'm unsure of something so want to pick the brains of anyone who might have
some further knowledge.

In an app that has a memory mapped file loaded, if it updates some mapped
pages and then does nothing else, then if I kill the app (e.g. with task
manager) the modified pages are written to disk, that is the updates are not
lost.

Further, if I run the same test but instead of killing the app, I just pull
power from the machine, the updates are lost and the files contents are
unmodified (or they might be if the OS paged the page out, but this is not
often the case).

Now my question:

If I run that same test, start the app, update some data in some mapped
pages and then (with the app running but idle) I do a system shutdown.

My understanding was that this operation would kill the processes (as I did
above with task manager) and then (eventually) finish the shutdown and power
off.

Because of this I have assumed that modfied pages would be written to disk
when I shutdown, much as they are when I just kipp a running app.

But is this true? I am seeing behaviour reported by a customer that suggests
a shutdown may actually kill processes more "brutally" and therefore might
not flush modfied pages to the mapped file.

I cannot find formal documentation on this anywhere.

If shutdown is more "dirty" in this scenario, then how can I detect shutdown
in my code? I'd ideally like to create either a handler or a dedicated thread
that gets signalled when shutdown occurs (this will be done inside our API
library DLL).

Handling it this was will mean we can handle it whether our library is used
in a windows app, a console app, a service etc and lets us handle it without
the user having to code for it.

Any info greatly appreciated.

H

[Francois PIETTE]

Post by Hugo <hugh<underbar>
But is this true? I am seeing behaviour reported by a customer that suggests
a shutdown may actually kill processes more "brutally" and therefore might
not flush modfied pages to the mapped file.
I cannot find formal documentation on this anywhere.

Shutdown is an orderly process. It is clean. No risk to loose data. I would
even say there is much less risk to loose data when properly shutting down
the computer (not by removing the power cord or clicking the hardware reset,
but using the shutdown procedure) than killing a process with task manager.

Post by Hugo <hugh<underbar>
If shutdown is more "dirty" in this scenario,

It isn't.

Post by Hugo <hugh<underbar>
then how can I detect shutdown
in my code?

Handle the message WM_ENDSESSION:
http://msdn.microsoft.com/en-us/library/aa376884(VS.85).aspx

[Hugo <hugh<underbar>]

Post by Francois PIETTE

Post by Hugo <hugh<underbar>
But is this true? I am seeing behaviour reported by a customer that suggests
a shutdown may actually kill processes more "brutally" and therefore might
not flush modfied pages to the mapped file.
I cannot find formal documentation on this anywhere.

Shutdown is an orderly process. It is clean. No risk to loose data. I would
even say there is much less risk to loose data when properly shutting down
the computer (not by removing the power cord or clicking the hardware reset,
but using the shutdown procedure) than killing a process with task manager.

Post by Hugo <hugh<underbar>
If shutdown is more "dirty" in this scenario,

It isn't.

Post by Hugo <hugh<underbar>
then how can I detect shutdown
in my code?

http://msdn.microsoft.com/en-us/library/aa376884(VS.85).aspx
--
The author of the freeware multi-tier middleware MidWare
The author of the freeware Internet Component Suite (ICS)
http://www.overbyte.be
.

Thanks Francois

That is more or less what I understood too.

Now here is next question, which might explain what is happening.

If the app maps a 2 GIG file, then populates say 1 GIG with data, this
equates to about 260,000 modified memory pages. Assume the box has lots of
memory (eg, 8 or 16 G) and that memory is under very little pressure.

Now when the reboot begins, it will go through the orderly termination and
flushing you described, but it will take X seconds for all of these modified
pages to be written to the disk (the media itself).

So there could be a weakness, in that the shutdown will wait only so many
seconds and could perhaps power off the box, while these pages are being
flkushed, resulting in not all modfied pages being written.

This sounds plausible to me, fixing it is a separate issue, but could this
happen?

H

[Hector Santos]

Our server make extensive use of memory mapped files (including 1-2
gigs sizes) and over time, we just decided that it was best to flush
record writes to resolve the occasional support issues where customers
where not gracefully shutting down or were doing machine shutdowns and
the server moved into a exit state or there was a brown out with UPS
failed, etc. In the past, speed was a consideration so always
flushing was not always desired but not as much today allowing us to
afford flushing record writes. The speed of computer and improvements
in Windows itself, make it more feasible to do. Bottom line result for
us: No more support calls.

If you can't afford flushing records with each write, then you might
consider a timer that will do so every X seconds or so.

Post by Hugo <hugh<underbar>
Thanks Francois
That is more or less what I understood too.
Now here is next question, which might explain what is happening.
If the app maps a 2 GIG file, then populates say 1 GIG with data, this
equates to about 260,000 modified memory pages. Assume the box has lots of
memory (eg, 8 or 16 G) and that memory is under very little pressure.
Now when the reboot begins, it will go through the orderly termination and
flushing you described, but it will take X seconds for all of these modified
pages to be written to the disk (the media itself).
So there could be a weakness, in that the shutdown will wait only so many
seconds and could perhaps power off the box, while these pages are being
flkushed, resulting in not all modfied pages being written.
This sounds plausible to me, fixing it is a separate issue, but could this
happen?
H

[Francois PIETTE]

Post by Hugo <hugh<underbar>

Post by Francois PIETTE
Shutdown is an orderly process. It is clean. No risk to loose data. I would
even say there is much less risk to loose data when properly shutting down
the computer (not by removing the power cord or clicking the hardware reset,
but using the shutdown procedure) than killing a process with task manager.

That is more or less what I understood too.
Now here is next question, which might explain what is happening.
If the app maps a 2 GIG file, then populates say 1 GIG with data, this
equates to about 260,000 modified memory pages. Assume the box has lots of
memory (eg, 8 or 16 G) and that memory is under very little pressure.
Now when the reboot begins, it will go through the orderly termination and
flushing you described, but it will take X seconds for all of these modified
pages to be written to the disk (the media itself).
So there could be a weakness, in that the shutdown will wait only so many
seconds and could perhaps power off the box, while these pages are being
flkushed, resulting in not all modfied pages being written.

Not sure I understand what you mean.
No matter how long it will take to write the data to disk, it will be
written in the shutdown process.
But if the user is impatient and turn the power off before the shutdown is
finished, then yes things will go wrong.

--
***@overbyte.be
The author of the freeware multi-tier middleware MidWare
The author of the freeware Internet Component Suite (ICS)
http://www.overbyte.be

[Hugo <hugh<underbar>]

Post by Francois PIETTE

Post by Hugo <hugh<underbar>

Post by Francois PIETTE
Shutdown is an orderly process. It is clean. No risk to loose data. I would
even say there is much less risk to loose data when properly shutting down
the computer (not by removing the power cord or clicking the hardware reset,
but using the shutdown procedure) than killing a process with task manager.

That is more or less what I understood too.
Now here is next question, which might explain what is happening.
If the app maps a 2 GIG file, then populates say 1 GIG with data, this
equates to about 260,000 modified memory pages. Assume the box has lots of
memory (eg, 8 or 16 G) and that memory is under very little pressure.
Now when the reboot begins, it will go through the orderly termination and
flushing you described, but it will take X seconds for all of these modified
pages to be written to the disk (the media itself).
So there could be a weakness, in that the shutdown will wait only so many
seconds and could perhaps power off the box, while these pages are being
flkushed, resulting in not all modfied pages being written.

Not sure I understand what you mean.
No matter how long it will take to write the data to disk, it will be
written in the shutdown process.
But if the user is impatient and turn the power off before the shutdown is
finished, then yes things will go wrong.
--
The author of the freeware multi-tier middleware MidWare
The author of the freeware Internet Component Suite (ICS)
http://www.overbyte.be

I think you are correct. I just ran some tests, if I call FlushViewOfFile
explicity then shutdown the system, the app does not vanish from the screen
until the flush finishes, the box then shuts down promptly.

If I dont call FlushViewOfFile, but just start a shutdown, the app's window
vanishes and the system displays the shutdown screen (spinning circle
"Shutting Down").

But this takes a while because the system is flushing to disk implicityly.

In each case the data (several gig in this test) is fully written to disk
before the box is switched off.

This is good.

However, this raises a new question for me, and I can't quite get a definite
answer from the MSDN docs, I'm posting this separately as "Detect If Shutdown
In Progress".

Thanks
H