While the importance of due process cannot be overstated, immigration violations are not generally crimes outside of a few specific areas. Removal proceedings are frequently not tied to any particularly crime, but merely unlawful presence, which is not a crime in its own right.
While you are correct in stating that an article III court generally is not required, the due process for immigrants, even those not present legally, is more complicated than just "check paperwork for legal status, act immediately". While in some cases expedited removal bypass the normal process, if a deportation is contested, due process still generally entails access to a hearing before an immigration judge (article II judge).
For criminal concerns regarding retroactive ITAR additions, yes. However, significant civil financial penalties if congress so wished could still be constitutional as the ex post facto clause has been held to apply exclusively to criminal matters starting in Calder v. Bull [1].
