Hi,
Last year, I did a similar handwriting recognition project, one of my approaches is that we used tesseract 4 to train handwriting. The dataset used includes the IAM handwriting dataset and the dataset we created.
Your requirements are very similar to this project I have done.
In addition, I also have experience in training, improving Tesseract for different languages (english, japanese), fonts.