When more than one interface is affected by changes in a single Puppet
run, it was easy to get into a state where the network on the node was
not functioning properly, since we were reconfiguring each interface
independently.  Examples include when changing from a single interface
to a setup with bonded interfaces, when configuring bridging (and IP
addresses move from the physical interface to the bridge interface),
or in general when routes are moved from one interface to another.  We
would reconfigure (ifdown+ifup) e.g. eth0, and then, much later, the
ifdown+ifup for bond0 would be executed.  Inbetween those, the network
was broken, and many Puppet resources (e.g. package resources) would
fail.

In most cases, a manual 'service network restart' after the failed
Puppet run, would bring the network up again, and the next Puppet run
would succeed, bringing the failed resources into sync, but it is an
annoyance to users.

We solve this by doing a single 'service network restart' after all
interfaces have been configured.  That way, there is no point in time
where some interfaces have been reconfigured and some not; either none
or all of them have the correct configuration.  (However, if Puppet is
interrupted during a run, it can leave the /etc/sysconfig files in an
inconsistent state; doing 'service network restart' or rebooting then
will give you a non-functioning network.)

The downside to this, is that *all* network interfaces, including
those not managed by Puppet, will be brought down for a short period,
even when only a single interface has its configuration changed.

This was originally commit 146bec6c3a26 in nsc-puppet-utils.