On June 22, 1941, right after the German invasion of the Soviet Union, the Soviets suddenly left Szarkowszczyzna and many Jews fled with them. Some of the refugees encountered German troops on the way and were forced to return. After the Soviet authorities left, the town remained without any rulers for several days. Farmers from nearby villages went on a rampage and ranksacked state storehouses and Jewish property. On June 27, 1941 German forces entered Szarkowszczyzna. In October of that year local Jews were incarcerated in two ghettos that were set up and Jews from nearby areas were also taken there. A total of about 1,900 people were imprisoned in the ghettos. Most of them were killed in a murder operation on July 18, 1942.
The town was liberated by the Red Army in March 1944.