From https://community.gaytor.rent/index.php?topic=68283.msg348756#msg348756 (thanks @bearbearbear)
To fix this issue, you need to take the special characters out of ALL of the file-names, by RENAMING the FILES.
ALL file-name characters must be normal ASCII characters, WESTERN ENGLISH FONT.
This error occurs when there is an unsupported character in one or more of the filenames in the torrent's contents. Generally, the most common character is the apostrophe. Please ensure that there are only regular letters or numbers, dots, spaces, or dashes in the filenames of all of the files in the torrent. Once that is done, try creating the .torrent file again and uploading the torrent again. The new upload should work once these characters are removed.