The prefetch buffer is constantly in use. As shown above, the instructions are stored in the Pre Fetch Buffer (PFB) and supplied to the processor from here.  Another speculative fetch begins as the ARM core is executing from the PFB and this data gets loaded into the data latch.  As soon as the last instruction in the Quad is executed, the data is transferred to the Pre Fetch buffer and the process starts again.