I've had an experiment with this and I think what you are trying to achieve, can't be done.
In my opinion you are just making hard work for yourself by using the Nashville system or Roman numerals.
It seems that BIAB does not take this into account with key changes. It also doesn't conform with the standard upper and lower case, unless I'm missing something. From what I can figure out, BIAB cannot modulate to different keys, the song itself is set to just one key.

Yes this is maybe a shortcoming but I don't personally see it as much of a problem. This is not composing software.

With that said, it's also not a problem unless this somehow alters they way it generates chords. Some say it does, I'm not so sure myself - either way, you need to understand one thing that's important...

"Key signature" and "Key" are not really the same thing. In this respect, I think BIAB is doing things correctly - it just isn't taking chord position into account as they all relate to the overall key that is set for the song.

If all you want to do is make the notation display correctly then just use chord names.

Example:

Key of C ... I, ii, iii, V = C Dm Em G
Key of D ... I, ii, iii, V = D Em F#m A

BIAB displays this as I IIm IIIm V which I find odd but maybe there is a setting I have overlooked.

Either way. Let's look at those chords in Roman numerals all in the key of C major (with BIAB interpretation in brackets).

C = I (I)
D = II (II)
Dm = ii (IIm)
Em = ii
F#m = #iv
G = V
A = VI

Personally I would have a harder time deciphering that than if I just looked at chord names, it just seems messy.

So back to using chord names and looking at what BIAB is doing.

4 Bars in the key of C
C Dm Em G

change key to D at bar 5
D Em F#m A

Now go to settings and change the key signature to D major at Bar 5 ... everything in the notation is correct and showing as it should.
BIAB just doesn't take that into account if you switch the chord display type. Just remember, key signature is a notation thing only - it does not relate necessarily to the home key.

That might seem a bit odd but it's not really. Most of these things are about keeping notation as tidy as possible. If you have a whole bunch of chords throughout a song that do not conform strictly to one single key (quite common) then notating everything as a change of key would become a mess for any reader. Much better to improve your theory and knowledge of chord function if you want to think in numerals. As I said, I personally think you're just making hard work of it.