It really depends on your family, but we’ve done late dinners before. My family doesn’t change time zones to go to WDW, but as a family, we typically eat dinner at 8:00 because of work schedules, so 9:00 or later isn’t much of a stretch for us.
You won’t be rushed out. You’ll probably get more attention. We did Chef Mickey’s late one night, and the characters didn’t have many guests to visit, so we got to see them multiple times.