When I left off last week I had written a small body of tests pertaining to the WebVTT’s spec for cue ids. I was fairly sure of myself when I made the pull request for them and I am still fairly certain of myself now, however it currently only has 1 review so far. I hope it will be reviewed further so I can get more feedback on my testing style, and so that they can be landed before merge conflicts arise.
Moving forward however, I made a few more tests during the week pertaining to the testing of our parser with other languages as well as how it might handle those languages when long strings are involved. Currently I think we should be proud to say none of the standard language tests I made had any problems passing with the parser (besides the recurring valgrind errors)! It should be considered quite an achievement to have a parser working just as well in English as it does in Thai, Hebrew or Chinese and it was great seeing the tests passing.
Unfortunately however, the parser’s ability to parse long strings might require some attention as none of my long string tests have passed the parser. After a short discussion with caitp in class and on IRC I discovered that the maximum length that the parser should be allowed to read before the string is truncated is 65536 bytes. This is probably alot more than it really needs to be, but it should technically suffice. The issue I’ve run into with the long string tests is that the string currently gets truncated on tests as short as 4000 utf-8 characters long. I suppose one of the issues that may be causing this is that a standard utf-8 character can reach up to 16 bytes long, however it confuses me a little that even my tests for US ASCII characters fails on lengths shorter than I was expecting (5822) as I was assuming those characters would be only 1 byte long. Perhaps I am misunderstanding something about utf-8, or I was looking at the wrong maximum length, however in the meantime some study will need to be done into why the errors occur.
In the future I plan to add a few more tests to WebVTT before I write out my 0.4 release. I have been looking at the WebKit tests and I think I have found one or two that may make good additions to our test suite. I also plan to look at the tests that Opera made for their implementation as well. Unfortunately however, I will probably need to look into adding these on Friday as my schedule is currently loaded with studying for exams and completing final assignments.