Cascaded emission from the biexciton state of a quantum dot results in polarization entangled photon pairs. However, modelling this system becomes challenging when photon emission is cavity-mediated due to the large Hilbert space and non-Markovian nature of its phonon-induced decoherence. Here, we introduce an algorithm that reduces the computational cost of the numerically exact process tensor method for non-Markovian dynamics simulations when the environmental coupling operator has degenerate eigenvalues, making calculations of the non-Markovian dynamics of large systems feasible. We compute the degree of entanglement of photon pairs generated by pulsed two-photon resonant excitation and find surprisingly good agreement between the numerically exact results and those calculated using the approximate polaron master equation method, permitting an efficient exploration of trends across system parameters.