To be honest, I have never done this. But you could try modifying this in packet.cpp:
#define AFSK_SPACE 2200
#define AFSK_MARK 1200
And setting the values to your custom tones of 1700 and 1300.
Luckily, the math to calculate the delay is all figured out for you. You will want to just adjust bit rate.
To adjust bit rate, you will need to modify packet.h:
#define SAMPLERATE 9600
#define BITRATE 1200
Set BITRATE to 600.
You may also want to adjust SAMPLERATE if you are having issues, as the code was originally written with the sample rate 8 times higher (oversampled) to measure 1200 baud. I looked over the code and it seems to be flexible, but we don't know. If it doesn't work, maybe try 4800 for the sample rate?