Patching for MMOs: A general approach

-=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- (c) WidthPadding Industries 1987 0\|280\|0 -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=-
SoCoder -> Article Home -> Presentation and Organisation

Afr0	Created : 13 February 2012 Edited : 15 February 2012 System : Cross Platform Language : Flash Patching for MMOs: A general approach Description of a general approach to patching clients
For the last few days (since Saturday) I've been working on a patching-system for Project Dollhouse. Originally it was going to consist of an FTP server and client, but after giving up on implementing FTP, I started writing a PHP-script (as suggested by Jay). What the script does is: Check the available manifests serverside against the client's version. If a manifest is found that is newer than a client's version, the script opens it and checks if it has a child and/or a parent. If it does, the child and/or parent is sent in a zipped file to the client. The script is called 'patch.php', and looks like this (Unix-version): <?php /The contents of this file are subject to the Mozilla Public License Version 1.1 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at https://www.mozilla.org/MPL/ Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. The Original Code is patch.php. The Initial Developer of the Original Code is Mats 'Afr0' Vederhus. All Rights Reserved. Contributor(s): ______________________________________. / include 'createzip.php'; if(isset($_GET['Version'])) { $ClientVersion = intval(htmlspecialchars($_GET['Version'])); $NewManifest = ""; if($handle = opendir(realpath('./patches/'))) { //Find out if there are any manifests newer than the client's version. while (false !== ($entry = readdir($handle))) { if(intval(str_replace('.manifest', '', $entry)) > $ClientVersion) { $NewManifest = realpath('./patches') . '/' . $entry . '.manifest'; break; } } if(empty($NewManifest) \|\| strcmp($NewManifest, realpath('./patches')) == 0) { header('Content-Description: No New Manifest'); header('Content-Type: text/html; charset=utf-8'); echo('<html><body><p>No new manifest!</p></body></html>'); } else { //Open the new manifest and check if it has a child, which means an //incremental update needs to take place... $FileHandle = fopen($NewManifest, 'r'); $Parent = fgets($FileHandle); $CurrentChild = fgets($FileHandle); $Path = realpath('./patches'); $Parent = str_replace('Parent=', '', $Parent); $Parent = str_replace('"', '', $Parent); $Parent = trim($Parent); $CurrentChild = str_replace('Child=', '', $CurrentChild); $CurrentChild = str_replace('"', '', $CurrentChild); $CurrentChild = trim($CurrentChild); fclose($FileHandle); //This is a compromise. Instead of checking for all childs, just check for the first one, //and let the client handle the other children (if any). if(empty($CurrentChild) !== true && empty($Parent) !== true) { $Parent = $Path . '/' . $Parent; $CurrentChild = $Path . '/' . $CurrentChild; $FilesToZip = array($CurrentChild, $NewManifest); $ZipName = './' . strval(rand()) . '.zip'; create_zip($FilesToZip, $ZipName); //Client should check the Content-Description header to figure out what to expect. header('Content-Description: Zipped File Transfer'); header('Content-Type: application/octet-stream'); header('Content-Disposition: attachment; filename='.basename($ZipName)); header('Content-Transfer-Encoding: binary'); header('Expires: 0'); header('Cache-Control: must-revalidate, post-check=0, pre-check=0'); header('Pragma: public'); header('Content-Length: ' . filesize($ZipName)); ob_clean(); flush(); readfile($ZipName); unlink($ZipName); exit(); } //There was a child, but no parent. else if(empty($CurrentChild) !== true && empty($Parent) == true) { $CurrentChild = $Path . '/' . $CurrentChild; $FilesToZip = array($CurrentChild, $NewManifest); $ZipName = './' . strval(rand()) . '.zip'; create_zip($FilesToZip, $ZipName); //Client should check the Content-Description header to figure out what to expect. header('Content-Description: Zipped File Transfer'); header('Content-Type: application/octet-stream'); header('Content-Disposition: attachment; filename='.basename($ZipName)); header('Content-Transfer-Encoding: binary'); header('Expires: 0'); header('Cache-Control: must-revalidate, post-check=0, pre-check=0'); header('Pragma: public'); header('Content-Length: ' . filesize($ZipName)); ob_clean(); flush(); readfile($ZipName); unlink($ZipName); exit(); } //There was a parent, but no child. else if(empty($Parent) !== true && empty($CurrentChild) == true) { $Parent = $Path . '/' . $Parent; $FilesToZip = array($Parent, $NewManifest); $ZipName = './' . strval(rand()) . '.zip'; create_zip($FilesToZip, $ZipName); //Client should check the Content-Description header to figure out what to expect. header('Content-Description: Zipped File Transfer'); header('Content-Type: application/octet-stream'); header('Content-Disposition: attachment; filename='.basename($ZipName)); header('Content-Transfer-Encoding: binary'); header('Expires: 0'); header('Cache-Control: must-revalidate, post-check=0, pre-check=0'); header('Pragma: public'); header('Content-Length: ' . filesize($ZipName)); ob_clean(); flush(); readfile($ZipName); unlink($ZipName); exit(); } //There was neither a parent nor a child. This should really never occur. else { header('Content-Description: File Transfer'); header('Content-Type: application/octet-stream'); header('Content-Disposition: attachment; filename='.basename($NewManifest)); header('Content-Transfer-Encoding: binary'); header('Expires: 0'); header('Cache-Control: must-revalidate, post-check=0, pre-check=0'); header('Pragma: public'); header('Content-Length: ' . filesize($NewManifest)); ob_clean(); flush(); readfile($NewManifest); exit(); } } } } else { header('Content-Description: Invalid Request'); header('Content-Type: text/html; charset=utf-8'); echo('<html><body><p>Invalid request!</p></body></html>'); } ?> --v A manifest basically dictates which files a patch/update consists of. They are ASCII files with their version as the name and a .manifest extension. Example manifest (generated by Manifestation): Parent="" Child="" NumFiles=48 &DLLs\Aries.dll MD5: ?5??????I?[/? DLLs\authlogin.dll MD5: ?????<?I?X??? 4DLLs\DBAppServiceClientD.dll MD5: "??(???N?$???? !dummy.jpg MD5: )8???Qc??R??c/ !dummy.tga MD5: O??????????NW& &dummy_8bit.bmp MD5: ?vu$"????By??? 'dummy_24bit.bmp MD5: ^p?&;????2?"j;) EP2.dat MD5: ?X???8??Y????w? $EP5Samps.dat MD5: ?J %?F[????? &DLLs\gimex.dll MD5: DV???O??IO> (HitListsTemp.dat MD5: ~l?{??JL,?2?9? .DLLs\HitMp3DecodeD.dll MD5: ????psCX?p'??2%6 &DLLs\ijl10.dll MD5: ?r?q.GcP?? 1DLLs\InternetServiceD.dll MD5: ? y?fU?\|[?<O )DLLs\libexpat.dll MD5: KW??(e0?????? &menulights.ini MD5: ?\2???????EU?? &properties.txt MD5: c????eh???%7?? 'rock_colors.txt MD5: i-??H??K??u ?pI? 'sand_colors.txt MD5: lm??\|j&1x+?o 'snow_colors.txt MD5: &?{H??<?V?q?fT "splash.bmp MD5: ?L???.g?N?????6 "Stings.dat MD5: ???~Z3/?v????? $template.lch MD5: {????8??Y???S?? )DLLs\TSOAnimD.dll MD5: B??si(JY?@4?p?? $TSOAudio.dat MD5: 8J?)????8?p?p?? .DLLs\TSOAudioViewD.dll MD5: ???.?}??!????? %TSOClient.exe MD5: ??s?$?#??8????A 2TSOData_datadefinition.dat MD5: #?!L??B? ???c} 6DLLs\TSODataServiceClientD.dll MD5: %?0????O??(5?{? 6DLLs\TSOEdithPluginClientD.dll MD5: ??:?@w?'k??m? (DLLs\TSOFont.dll MD5: ???B?mu??L?? R tsofp.va MD5: i+<????????X DLLs\TSOGimexD.dll MD5: ??W? ???????R ,DLLs\TSOGraphicD.dll MD5: ?=gdhV??>ro??? 8DLLs\TSONetServiceSimClientD.dll MD5: !???m??>`<??+? ? ,DLLs\TSOPersistD.dll MD5: `dN7??c?i&?N? 1DLLs\TSOPersistToolsD.dll MD5: ??u?\|.????E? 6DLLs\TSOPresentationLayerD.dll MD5: K?7==2cl?U]N? 2DLLs\TSOServiceClientD.dll MD5: ?k9?7??K??';^ ? 4DLLs\TSOSimulatorClientD.dll MD5: ?.?????<??????? !tsov2.dat MD5: Pk?`?'?p."??` 1DLLs\TSOViewerClientD.dll MD5: ?H???r?\|yYlI +DLLs\TSOWinAppD.dll MD5: ??{`?{?c??1k?o?? 0DLLs\TSOWorldClientD.dll MD5: ?v??N?)???J? "tuning.dat MD5: mx???HW?????r.n +tuning_hardcore.ini MD5: ???%?;+???????!A $userinfo.ini MD5: ?9\.J????F?dX?? version MD5: z?g??????8 --v As you can see, this manifest has a parent, which is the next patch/update released. It has no child because it is the first patch released. Then it specifies the number of files to follow. Each file consists of a full path + filename, followed by a checksum. The checksum is used by the client to verify that a file was successfully downloaded. What's important to note is that the path to a file is assumed to be where the file will be located on the client's side. Serverside, files are always assumed to be located at https://www.webserver.com/patches/PatchVersion/File1.file* In addition to 'patch.php', the patching-suite for Project Dollhouse also consists of a tool for generating patches called Manifestation and another script called 'getmanifest.php'. Manifestation can be downloaded here. Here is the design for how a client should behave in order to ensure it is fully up-to-date (since this article is supposed to be general-purpose, I won't delve into the sourcecode for the client here); Step 1: Client connects to webserver/patch.php patch.php sends the latest manifest if one exists that is newer than the provided version. Each manifest must specify a child, optionally a parent. If the latest manifest has a child, this child will be zipped together with its parent. Step 2: The client sorts the downloaded manifests in descending order, and check if the newest one has a parent. Step 3: If it does, the client downloads it from webserver/getmanifest.php This process should be repeated until the client downloads a manifest that doesn't have a parent. If the initial parent doesn't have a parent, this step can be skipped. Step 4: Client checks the oldest manifest to see if it has a child that is newer than the client's version. If it does, the client downloads it from webserver/getmanifest.php This process should be repeated until the client downloads a manifest that EITHER doesn't have a child, OR doesn't have a child that is newer than the client's version. Step 5: Client reads all the manifests in turn (ascending), downloading each file and checking it against the checksum in the manifest. If a file isn't properly downloaded, the user should be informed and given the choice to redownload. Once all the files in a manifest has been downloaded, the patch is applied and the process is repeated from step 5. --v Now to the last step in this general approach to patching; fetching manifests. Arguably, a patch-client could do this manually, but I feel it would be a little safer to have a script do it instead. So I wrote 'getmanifest.php'; <?php /The contents of this file are subject to the Mozilla Public License Version 1.1 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at https://www.mozilla.org/MPL/ Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. The Original Code is getmanifest.php. The Initial Developer of the Original Code is Mats 'Afr0' Vederhus. All Rights Reserved. Contributor(s): ______________________________________. / if(isset($_GET['Manifest'])) { $RequestedManifest = htmlspecialchars($_GET['Manifest']); $FoundManifest = ""; if($handle = opendir('./patches/')) { //Find the requested manifest... while (false !== ($entry = readdir($handle))) { if($entry === $RequestedManifest) { $FoundManifest = './patches/' . $entry; break; } } if(empty($FoundManifest) \|\| strcmp($FoundManifest, './patches') == 0) { header('Content-Description: Invalid Request'); header('Content-Type: text/html; charset=utf-8'); echo('<html><body><p>No manifest by that name exists!</p></body></html>'); } else { header('Content-Description: File Transfer'); header('Content-Type: application/octet-stream'); header('Content-Disposition: attachment; filename='.basename($FoundManifest)); header('Content-Transfer-Encoding: binary'); header('Expires: 0'); header('Cache-Control: must-revalidate, post-check=0, pre-check=0'); header('Pragma: public'); header('Content-Length: ' . filesize($FoundManifest)); ob_clean(); flush(); readfile($FoundManifest); exit(); } } } else { header('Content-Description: Invalid Request'); header('Content-Type: text/html; charset=utf-8'); echo('<html><body><p>Invalid request!</p></body></html>'); } ?> --v This script is invoked by a client as such: https://www.webserver.com/getmanifest.php?Manifest=Manifestversion.manifest It will then send the requested manifest to the client, or an error if the manifest wasn't located. This might not be the most effective, safe, or even reliable way of patching clients, but it works. If you have any questions, post them below.

Comments

Patching for MMOs: A general approach

Comments