The decision to hand over the boys aged one and three to their grandmother in Vienna was made after positive DNA results and a court granting her custody, according to foreign ministry spokesman Peter Guschelbauer.
"We have decided to bring back the two orphans, and preparations have started... It is the first repatriation of children from this region," he told AFP, adding that the process could take several weeks.
The children are now in the crammed Kurdish-run Al-Hol camp in northeastern Syria. Their Austrian mother, who went to join IS in 2014 when she was 15 years old, is believed to have died.
Guschelbauer said at least three other children could be repatriated later.
Last week, Kurdish authorities in northeastern Syria handed over to Germany four children from IS families, all of them under 10 years old.
Another dozen children of alleged jihadist fighters have been repatriated from Iraq to Germany since March.
France and Belgium have also brought a handful of orphans home, while the United States last year repatriated a woman with her four children.
Kazakhstan, Uzbekistan and Kosovo have repatriated dozens of women and children.
IS overran large parts of Syria and Iraq in 2014 and proclaimed a "caliphate" there, but the jihadist group has since been ousted from those territories.
Follow us on Twitter: @The_NewArab