Genetic drift and gene flow
Founder events and bottlenecks can change allele frequencies and reduce variation because survivors or colonists are unrepresentative samples. Neutral alleles can become fixed or lost without a fitness advantage. Increasing population size afterward does not automatically recreate lost alleles. Migration can introduce variation locally and often makes connected populations more similar. Inbreeding changes genotype proportions by increasing homozygosity, but alone need not change allele frequencies.
A storm leaves five randomly surviving beetles. Their alleles determine the rebuilt population, even if the survivors were not better adapted to storms.
Drift is not a synonym for selection. Ask whether reproductive differences were associated with a trait or resulted from sampling chance.