We cound try one or other of the remediations mentioned in the Firebug warning message I quoted above,

I figured it out, though I really should have been busting my ass getting these project SRS and SDD requirements finished for work before I leave for Houston tomorrow.

It ended up being a matter of php handling UTF-8 just fine, but that data enroute to the database was borked. The database itself held the data fine, as witnessed by old posts which display unicode correctly, the problem was inserting data into the database from php.

Did some more config comparisons and the mysql client needed to be in latin1 charset, while the database itself needed to be in utf8 charset. Whatever, sounds backwards but that's what it wants. So it just happened that php5.3 handled the low unicode byte eventhough the actual connection to the database wasn't what the forum wanted. Pure fluke. Now that the connection encoding is latin1, it seems everything's working now.

I moved the forum back to PHP 5.5, tested some stuff and it looks like unicode is working again:



キール「エヴァ３号機は予定通り、第９使徒の依り代としての役目を終えた。」
「予定通り、使徒とエヴァの融合は可能であったな」
「そして、エヴァを触媒とし、我ら人類との融合の可能性も見いだせた」
À Á Â Ã Ä Å Æ Ç È É Ê Ë Ì Í Î Ï Ð Ñ Ò Ó Ô Õ Ö
×
Ø Ù Ú Û Ü Ý Þ
ß
à á â ã ä å æ ç è é ê ë ì í î ï ð ñ ò ó ô õ ö
÷
ø ù ú û ü ý þ
ÿ
Γραμματοσειρές
તમે એક સંત છો
คุณนักบุญ,
كنت قديسا،
Είσαι ένας άγιος,
אתה קדוש, אורנט

Also, there was a short site outtage when I was messing with all of this stuff. Tried to restart mysql with the new settings and there was some zombie process holding onto the socket and it wouldn't let me restart it, and I couldn't kill the process because, well, it was zombie. Had to reboot the server so there was maybe 15-20 minutes of downtime.

You said earlier that stored posts were OK, but I'm seeing otherwise. Here

J�nglinge = Jünglinge
ma�tre = maître

Ornette wrote:It looks like some unicode in usernames didn't make it entirely unscathed eventhough I was explicit in both the dumps and restors. Fortunately, post body unicode, or at least the Japanese, seems to have been ok: https://forum.evageeks.org/post/439425/Eva-2XX-Storyboards-vs-Final/#439425

A couple of glyphs on that page now seem to have become corrupted, but the rest is OK.

May have been because the DB was imported while mysql connection was in utf-8 mode, and now when it's in latin1, those glyphs doesn't translate correctly. I guess it's possible to go through an old DB dumb and re-insert those posts. A bunch of user sigs seem to also have been affected.

يمارس الجنس مع اختبار
test atdırmaq
тэст ебать
тест дяволите
测试他妈的
testi helvettiä
ტესტი მივხვდი
σκατά δοκιμής
Fuck tès
לעזאזל מבחן
परीक्षण बकवास
próf ríða
ag fuck tástála
テストの性交
테스트 씨발
tests izdrāzt
bandymo velnių
دمار از روزگارمان درآورد آزمون
kiểm tra quái

Ok, so to sum up some of the extra curricular stuff that's been happening:

1. Now that the transfer to and from the database is tip top, some of the data that was inserted into the database before today and after the server migration had wonky utf-8 / latin1 encoding.
2. Additionally, some older posts that had some latin1 was wonky because when the database was migrated, the transfer encoding was wrong so those got wonked out as well
3. Now that the transfer encoding is fixed (I hope), I went through a pre-migration db backup and pulled out the post data that had non-ascii characters in it and updated them. That means if you edited a post in the last few days and it had non-ascii characters in it, I probably blew away your changes.
4. Usernames and signatures also got wonky because of all of this, for the most part, they're only certain glyphs that suffered.
5. I went through the same process to update the signatures but rolled back a bit too far, so people who edited their profiles recently will have to do it again, sorry.
6. I know at least one new user suffered from this (and I think that's the only one), the roll back killed any recent new sign-ups. I was going to cherry-pick a roll forward but it looked like the only one that had made any posts already re-registered so I didn't bother.

So again, your profiles got rolled back to a few days ago, and unfortunately, that includes avatars as well. Didn't think avatars was included in the script but whatever.

Painful - but a good job well done. Thanks!

Our host will be doing some Xen updates so there will be a short downtime on 2015-03-08 4:00:00 AM UTC, so that's like 11pm Saturday night EST. Not exactly, the best choice of time for the forum, but considering the host itself has had our VPS up with zero downtime for over 2 years straight, it's a small bother.

Just a heads up to the forums that some stuff we've been working on in the background for the past several months is going to be going live in the near future; this may mean some service interruptions., but should leave us with an improved experience for all.

We'll give more detailed announcements when changes are about to happen.