• Enhancements:
  • Upgrade to Smarty 3. Some templates have been reorganized to take advantage of Smarty 3′s hierarchical template features
  • RequireJS is now used to serve javascript. RequireJS is a module loader that has allowed me to restructure the javascript code into clean reusable modules. My javascript code before was… let’s just say gross. Using RequireJS does have some other benefits too:
    • RequireJS loads modules asynchronously and in parallel. Many browsers still load javascript files from the web server one at a time, blocking each load until the previous one has finished loading. This improves page load time.
    • The RequireJS javascript minification process (which I do for all packaged releases) will combine all modules for a page and their dependencies into a single file before compression, which reduces the number of separate files the user’s browser has to load.
  • GitPHP can now read project-specific config options from the git config in the repository directory itself (e.g. gitphp.git/config), to allow project-specific settings to carry across multiple gitphp installations. This file can be edited directly or accessed using the ‘git config’ command. The section is [gitphp], and the config values are each of the project specific settings currently supported in projects.conf.php. See the comments in projects.conf.php.example for more information. The project-specific config file will override global gitphp settings in gitphp.conf.php, but install-specific settings in projects.conf.php will override the project-specific config file.
  • GitPHP can now use the Google Libraries API to serve the jQuery library, by setting the ‘googlejs’ config value. Using the Google Libraries API allows you to offload serving that library from your web server, and allows users to benefit from a single cached file for all sites using the Google Libraries API. The library is served from Google’s servers, which means this won’t work if you’re running on an intranet without outside internet access.
  • Signed-off-by lines in commit messages and PGP blocks in tags are styled differently to differentiate them from the actual commit/tag message
  • The shortlog now displays abbreviated hashes for each commit. The abbreviation length is read from the project’s git config file (core.abbrev setting), defaulting to 7. By default, abbreviated hashes are not checked for collisions. A gitphp config setting, ‘uniqueabbrev’, has been added which will turn on collision checking when abbreviating hashes. Note that this is performance intensive because it needs to search every hash in the project, which is why it’s off by default. You might be better served just increasing the minimum abbreviation length.
  • Japanese translation, thanks to Ishikawa Mutsumi
• Fixes:
  • Fix a collision when multiple users were downloading an uncached version of the exact same snapshot at the exact same time
  • Fix direct line links on non-GeSHi blob pages, thanks to Steve Clay
  • Fix order of shortlog/log commits when a branch is rebased
  • Fix trimming of multibyte commit messages in shortlog, thanks to Ishikawa Mutsumi
  • Fix handling of git’s commit encoding header in commit messages
  • Fix handling of non-ASCII filenames in tree view. Based on a fix by sh2ka
  • Re-enable whitespace trimming to decrease the size of HTML files served. Was accidentally disabled during the big rewrite a year ago
  • Fix a potential XSS vulnerability

Smarty 3 was a difficult transition that’s been in the works for a long time. Smarty 3 was a complete rewrite from smarty 2 – and as I certainly found out during my big GitPHP rewrite, a complete rewrite tends to introduce slight unintentional functional changes. And for Smarty, as a library, those slight functional changes manifest themselves as bugs in the consuming app. I’ve spent a long time hunting Smarty 3 bugs in GitPHP using every single possible configuration combination I could think of. I believe I’ve gotten all the ones I can find, but I can’t possibly reproduce every single setup and git repo out there – so if you run into issues, by all means let me know.

Smarty 2′s compiled templates and cached files are incompatible with Smarty 3, so for safety I would suggest deleting the compiled templates in the template_c/ directory, any cached files in the cache/ directory, and bouncing Memcache if you use it. Smarty 3′s template compilation takes longer than before due to the hierarchical templates, however this will only happen the first time a page is loaded after deleting the contents of templates_c, you won’t have to do it again.

The release is on the GitPHP page and bugs can be filed on the bugtracker.

• Enhancements:
  • Move a lot of the data loading to using raw php file parsing instead of relying on the git executable. This causes an enormous performance boost, especially on webservers that have very expensive process forking (I’m looking at you Apache)
  • Add a ‘compat’ config option that can be specified globally or per-project, to fall back on the old method of loading data if you run into issues with the previous enhancement. (PHP will not process repository packfiles larger than 2GB)
  • Add a ‘largeskip’ config option. Using raw PHP data loading, we have to walk commits down the log as you page down earlier and earlier in the history, which has decreasing performance as you go back. This determines at what threshold gitphp will instead just use the git executable.
  • Display merge commits in the shortlog with grayed-out titles, thanks to Tanguy Pruvot
  • Subdirectory snapshots now include the subdirectory as part of the archive name
  • Support for the xdiff php extension. If you have the xdiff php extension installed, that will be used for diffing, which is faster and completely eliminates the need to have a separate diff executable or temp dir configured.
  • Support for loading the project list from an SCM-Manager config. Only projects marked public are loaded. Based on work done by Craig Sparks
  • Add debug info when searching directory for projects. If you aren’t seeing certain or any projects appear when letting GitPHP find all projects in the project root, turn on the ‘debug’ config option to see more about what it’s doing.
  • Benchmarking info is now turned on separately from debug info, with a new config option ‘benchmark’
  • Cache list of projects if the object cache is turned on, instead of searching the project root every time. Based on work done by Tanguy Pruvot
  • Archives are now delivered incrementally to the user’s browser, instead of being loaded entirely into memory first. This avoids PHP out-of-memory errors when trying to snapshot a very large project. Note: as a result of this change, snapshot tarballs are no longer cached to Memcache, even if Memcache support is turned on. They will always be cached to the cache directory on disk.
  • Minify CSS for performance
  • Support tags pointing directly to blobs. This isn’t commonly done but can be used to embed something like a GPG key in a repository
  • Allow displaying a website URL for a project. Because this can’t really be automatically calculated the way clone/push urls can, this is a per-project setting only.
• Fixes:
  • Fixed how the default diff executable is determined if not specified in the config
  • Fixed issues diffing when the temp dir had spaces in the name (common on Windows)
  • Fixed issues diffing on windows when the temp dir didn’t have a trailing backslash
  • Avoid floods of warning messages when the fileinfo magic database is incorrectly compiled
  • Avoid warning messages when the project doesn’t have a description file. (sometimes happens with repositories created by third party software other than the standard git program)
  • Fixed some display issues on the project list with owner/age/links columns wrapping too much when the project has an extremely long description
  • Avoid warning messages when listing all projects in the projectroot and the webserver user doesn’t have read access to one of the directories. Based on a fix by Justyn Shull
  • Fixed issue where using the diff link by a single file on the commit page showed the wrong commit at the top of the diff

As always, the release is on the GitPHP page and bugs can be filed on the bugtracker. If you need support you can always email me, or if you would prefer a more public discussion, you can use the forums.

Q: You’ve had a smarty 3 branch in progress for a while now. Why haven’t you merged it yet? Smarty 3 is perfect and amazing!!!1!eleven1one
A: Not quite. Smarty 3 is a complete rewrite and, just like the complete GitPHP rewrite a year ago, has some bugs and some differences in behavior. These changes in smarty 3 actually broke some of the functionality in GitPHP. I’ve been making changes to accomodate new behavior, and doing hacks to work around smarty 3 bugs, and I’m still not convinced I’ve found all of them yet. Additionally, Smarty 3′s cache infrastructure is completely different from Smarty 2, which renders all custom cache code useless. In particular, this means that GitPHP Memcache support does not work at all with Smarty 3. The upgrade to smarty 3 is something that makes development easier but has no real discernible effect for end users. I refuse to do an ‘upgrade’ that will negatively impact end users, especially when that negative impact is lost functionality.

The majority of the work is done.  I’m going to be merging it into master relatively soon, with the intent of including the changes in the next release.  Since this is such a fundamental change to the way data is loaded, I’m providing a configuration option to fall back to using the git executable (like GitPHP has always done) in case there are issues, and I’m considering providing this option on a per-project basis.

Update 7/17: This is merged into master. Any beta testers willing to test the new loading code on their repositories by running from git master would be appreciated… maybe we can knock out any major compatibility issues early.

Q: Why?
A: Several reasons:

1. Compatibility.  The less calls I have to do to the shell, the less I have to worry about the differences between Linux and Windows, and breaking one or the other (usually Windows, since I develop on Linux)
2. Performance.  Every shell execution is a fork() off of the webserver’s process.  When you run lots of shell commands one after another, fork()-ing the web server process each time, the delay is noticeable.
3. Security.  While the good security practice would be to make your git repositories read-only to the webserver user, I bet very few people actually do that.  Passing user input down to a shell command is always a dangerous thing to do, and potentially exposes your server to hackers.  Unfortunately, I’ve always put functionality above security when calling the git executable, and I’d like to change that.

Q: Why didn’t you use Glip?
A. I experimented with using Glip.  In doing so, I found a couple things:

1. Glip is incomplete.  It doesn’t load all of the objects that can exist in a git project, and is also missing a number of features I would need for it to replace the git executable.
2. Glip provides its own API to access objects in a git project – Blob, Commit, Tree, etc.  These objects were very similar to the git object representations I had created for GitPHP, but the API was different enough to make them incompatible.  In loading data using Glip, I found myself creating Glip objects, manually picking out values off of the properties just to stuff them into my own properties, and discarding the Glip objects.  Not only that, but the way it stored data in properties was different enough from my git objects that I ended up having to do redundant conversions.  Glip would load data in git’s internal representation, convert them to php’s internal representation, and then I would extract those values and convert them back into git’s internal representation for use in my objects.  It just felt like a really redundant amount of glue code.  I have no doubt that Glip would be great if you were writing a project from scratch based off it, but ripping apart and rewriting all of GitPHP’s model code is not something I’m looking to do anytime soon.

Q: Does this mean the git executable is no longer required?
A: Right now the git exe is still required.

Q: Does this mean we may not need the git executable in the future?
A. I don’t want to say never, but this is unlikely.  While loading data from plain git objects is significantly faster in raw PHP, this also means that the processing has to be done in PHP.  There are a number of cases where large amounts of processing in PHP actually make it slower than just biting the bullet and calling the git executable:

1. The git rev-list command (used for the shortlog/log) has a –skip command, which is used to skip a certain number of commits down in the log.  This is used for the next/prev paging in the log in GitPHP.  In raw PHP, we don’t have this option – we have to walk all the way down the commit log ourselves.  So in PHP, if you want to get page 5 of the log (commits 401-500), you have to walk the log and load commits 1-500, and discard the first 400.  So you can imagine that it gets really slow when you get down to page 21, and you have to load the first 2100 commits and discard the first 2000.  And walking the log isn’t just following each parent link – it’s any commit reachable from the tip, which includes all merged branches and any of their reachable commits.  The current implementation actually uses raw php for the early pages of the log, but when skipping a significant number of commits, falls back on the git executable in order to keep performance reasonable.
2. Searching for a commit, committer, or author requires loading every single commit in the history.
3. Grepping inside files requires loading up the contents of every single file in a tree and searching every line.
4. Getting the history of a file requires reading every single commit in the entire history and reading each commit’s tree to see if it touched this file.  That also doesn’t take into account things like detecting renames.
5. Getting the blame of a file requires getting the entire history of a file and diffing every commit with its parent to figure out the changes to the file.
6. Diffing a file would require me to write my own diff algorithm (non-trivial), or requiring an external php extension like xdiff, which I don’t want to do.

Q: Are there any additional requirements?
A: You’ll need Zlib support in your PHP, since git objects are gz-compressed.

Q: Are there any limitations?
A. The new loading code won’t read packfiles larger than 2GB.  This is a limitation in Glip, too.  It’s because php’s fseek() (or actually most operations in php) top out at 2GB (2*1024^3).

I’ll update this post with any other FAQs I can think of.

• Chinese translation – thanks to seefan
• Side-by-side diffs for blobdiff and commitdiff – based on work by Mattias Ulbrich and Tanguy Pruvot
• Fix clone and push urls using ssh colon notation – thanks to mdevilz
• Allow specifying hashbases by branch/tag name in URLs – previously you were required to specify the entire ref name, eg hb=refs/heads/master. Now you can just specify the head/tag name, eg hb=master.
• Fix crash in blobdiff_plain when specifying hashbase without a hash – this prevented you from using urls with just a filename and hashbase id or ref name, which was supposed to find the right file hash automatically

Release is on the GitPHP page and bugs can be reported on the bugtracker.

• Degrade gracefully when the system doesn’t have posix functions
  Previously there was a bug where the page crashed when trying to display the project owner on systems without the php posix module installed (windows systems, and several linux distributions). If you don’t have this module then the owner still won’t display unless you set the gitweb.owner config value or override it in projects.conf.php, but at least the page won’t stop rendering it.
• Make the language setting a permanent cookie
  Previously, the language cookie was a session cookie, so when you came back to the site you had to change your language again. Now it’s a permanent cookie with a 1 year expiration.
• Escape HTML in the title header
  Previously, commit messages weren’t escaped in the title bar (the gray bar with the commit message at the top of the page, below the nav), so commit messages with HTML in them caused weird things to happen. Now the HTML displays properly
• Add attribution and link back to GitPHP page in the footer
  This is by request, I swear. If you don’t like it, I don’t mind if you delete it from the footer and go back to the old appearance.
• Run javascript project livesearch after the user stops typing
  Previously the javascript livesearch on the project list ran after every letter you typed. This caused a severe performance hit on larger project lists, since the browser would churn through the huge list of projects many times in succession. Now it runs after the user stops typing.
• Remove containing tag from object cache
  Previously an object’s containing tag was stored in the object cache. The containing tag is actually not immutable data, and so there are certain cases where caching this would cause the display to get out of sync. Now, this is no longer cached. I would suggest clearing your cache after upgrading to be safe.
• Improve performance of projects with lots of tags or heads
  Previously there was a severe performance problem with projects with many tags or heads (hundreds). This was because it was attempting to load all tags and all their data into memory. For example, the git repo for the git program has almost 400 tags, and took almost 2 minutes to load any page. Now, it only loads the data for the tags it needs so it’s a lot better.
• Clean up stylesheets
  Previously the stylesheet was a mess and was really difficult to read. I’ve cleaned it up, organized it, and broken it out into two separate stylesheets – a functional stylesheet (gitphp.css) which contains declarations needed for gitphp to layout and work properly, and a look and feel stylesheet (gitphpskin.css) which customizes the colors and styling. The stylesheet config option now controls the location of the skin stylesheet – so you can make a copy, customize the look and feel, and point the stylesheet config option at the new skin stylesheet. The config option has backwards compatibility to handle the fact that the stylesheet name changed but people may have the option pointing at the old single stylesheet from previous example configs.

As always, the release is available on the GitPHP page and bugs can be reported on the bugtracker.

Full changelog:

• Enhancements:
  • Atom feed support, thanks to Christian Weiske
  • Error pages now return proper HTTP error codes to avoid search engine indexing
  • Users can now choose the file format of snapshot they want, based on what’s supported by the system. The config value still controls the default for non-javascript users.
  • Directories in the config file no longer require you to specify the trailing slash
  • Overriding project settings (category, owner, description, clone url, etc) can now be done for all project listing methods – directory list, file list, and array list. Previously it could only be done with the array
  • The tree view can now be drilled down using AJAX
  • Memcache support
  • Support for linking bug numbers in commit messages to a bug tracker
  • Search box to search projects on the project list. This is a live search if you have javascript
  • Object cache, for caching immutable git data – more info on what this is and why you want it is here
  • Javascript is now minified to decrease its size
  • Clone/push urls on the project page are now links, thanks to Cory Thomas
  • Project owners are now read from the git config value gitweb.owner if set, thanks to Cory Thomas
• Translations:
  • Russian, thanks to Aidsoid
  • German, thanks to Andy Tandler
• Bugfixes:
  • Fix issue where commit tooltips didn’t escape HTML characters correctly
  • Project ages on the project list page now use a more accurate method to get the age
  • The default tmpdir, if not specified in the config, is read from the system/php config rather than hardcoding it. This also fixes an issue where the default tmpdir for windows was incorrect
  • Fix the default git binary for windows x64 installs
  • An error message now displays when the diff/git executable isn’t found or doesn’t work, rather than just failing silently

For those who are wondering about this post about a backwards incompatible change – the change to specify overrides for all project list formats does change the project config file (projects.conf.php) format slightly, but the code is backwards compatible and will continue to use your old project config file until you adjust it for the new changes.

As always, the release can be downloaded from the gitphp page and bugs can be reported on the bugtracker.

The biggest change is that my homebrew translation framework that appeared in 0.2.0 was ripped out and replaced with gettext. I apologize if you already started translating, but gettext is much easier to work with, I promise. I’ve provided a short starter HOWTO in doc/TRANSLATING. As always, you can send a translation to me for inclusion in the official release.
If you’re running gitphp from the git repo instead of the tarball, you will need to run util/msgfmt.sh from the gitphp base directory after cloning the repository to compile the gettext MO files, or else the translations won’t work. The tarball releases have this done already and will work out of the box.

Language can also be selected on the fly by users now, using a drop-down menu. The default setting for this for a first-time visitor is the browser’s preferred language if supported, followed by whatever’s specified in the gitphp config, followed by English as the final default. Check it out at the live copy.

Also, this is the first release of gitphp that provides another language besides English: gitphp is now available in French, thanks to Zaran.

The other notable enhancement is the ability for users to select arbitrary commits on the log or commit pages, and diff them. Each commit in the log or shortlog shows a “select for diff” link. Click this by a commit to select it. The selected diff will appear in the very top header of the log or shortlog, with a partial commit message (with the full commit tooltip), and the option to deselect it. The “select for diff” links by commits will now change to “diff with selected” – click this to do the diff. (The link by the currently selected commit will change to “deselect”) This selection persists across pages of the shortlog/log, so you can diff across two different pages. The selection will also persist when switching back and forth from log to shortlog and vice versa, although going to a non-log page will discard the selection. The implementation is original, but I’d like to thank Calen Pennington for some of the UI designs.

Here’s the full list of changes:

• GitPHP now uses gettext for internationalization. I’ve also provided a short howto in doc/TRANSLATING
• Language can now be changed on the fly by the user using a drop-down menu in the upper right. The initial default for this is the user’s preferred language from their browser. If there is no translation for the user’s language, it will fall back to the locale in the config, then to English as the final default. But of course, the user can always change the language whenever they want. The user’s language choice is remembered by a browser cookie.
• French translation, thanks to Zaran
• On the log and shortlog pages, arbitrary diffs can be selected and diffed. You can select one commit, then diff with any other. Your selected diff will persist across pages, in case you want to diff between two different pages. The selected diff, if there is one, will appear in the top header, above the log/shortlog. This selection will also persist when switching from the shortlog to the log and vice versa.
• New pieces of metadata can now be specified for a project using the advanced project array format (in projects.conf.php): description, owner, clone url, and push url. Specifying any of these for a project will override that data for that project only.
• Tree view and commit view now provide a “plain” link by blobs to go right to the plaintext view of that blob
• Added a file history link to the blob page
• To make things a little easier to track, the home text template that configures the header of the project list (templates/hometext.tpl) is no longer provided in the tarball. The system will show the default hometext, but you can create templates/hometext.tpl with your content to customize the header. Therefore, this is backwards compatible for everyone who already has a customized hometext.tpl template.
• The “projects” link in the upper left that takes you to the main project list can be customized. Please be advised that if you override this, you’re also hardcoding the language that it appears in.
• Fixed a bug where search wouldn’t work on git < 1.5.3
• Fixed a bug where some dates wouldn’t show up correctly on windows
• Fixed a bug where an exception would be thrown if no config file existed
• Fixed bugs where GitPHP would crash when an empty project with no commits was being loaded
• Fixed a bug in reading projects from a list file, where owners with multiple words in the name would only display the first word
• Fixed a bug on the commit page, where the history link by a deleted file wouldn’t work
• Fixed a bug where the javascript commit tooltip would appear out of the browser window for a commit close to the bottom of the page
• XHTML validation fixes, thanks to Christian Weiske
• Minor visual fix where source files in commit diffs were missing the “a/” prefix (but destinations files still had the “b/” prefix)

Get it on the GitPHP page.

As always, anything – bugs, enhancement requests, etc – can be reported on the Mantis bugtracker. I’d like to give a big thanks to the people that have been using the bugtracker already; it’s been a huge help in keeping me and my development organized.

I’ve also created a page for GitPHP on Ohloh, if you’re interested.

I’m releasing it as 0.2.0 – I figured that the architectural rewrite is significant enough to warrant some kind of larger version bump, but since it doesn’t look and behave all that differently to end users, a major version bump isn’t necessary.

Here’s what I came up with for the changelog, although as a total rewrite from scratch, there could very well be other “features” and changes that made it in that I’m not thinking of, just by virtue of the entire codebase changing:

• Major architectural rewrite – end users aren’t heavily affected, but developers will appreciate it. On the other hand, PHP 4 is no longer supported
• Much more simplified config that works on an override strategy – so your config file can be much cleaner
• Can fetch snapshots of a subtree (subdirectory), rather than the entire tree – thanks to Tejas Dinkar
• When listing out projects in the projectroot (instead of explicitly listing them in the config), can optionally respect the git-daemon-export-ok magic file used by git-daemon to include/exclude projects – thanks to Calen Pennington
• When listing out projects in the projectroot (instead of explicitly listing them in the config), projects in subdirectories will be automatically “categorized” by their directories – thanks to Calen Pennington
• AJAX javascript tooltips to display the full commit message when hoving over a commit link, and the full tag message for a tag object
• Use AJAX javascript to load blame info right into the blob page, instead of posting to a separate page
• Internationalization support, although we don’t have any translations yet
• Blame page now supports syntax highlighting
• New way of specifying categories for projects – in preparation for more metadata to be added to projects
• The little colored ref badges now have different colors to distinguish between heads and tags
• File sizes by blobs in the tree view
• File searching shows the matching line number

In theory this will work as a drop-in replacement for previous versions – no changes necessary. However, because of the newer and cleaner config system, it might be worth it to run through your configuration again using the example and copying from the defaults file, just to clean up the amount of stuff you have to look at in your config file. Also, if you were using the method of defining projects as an array to specify categories, you might want to have a look at the new way of specifying the categories in projects.conf.php if you have a chance. You don’t gain anything from changing to the new format now so you don’t have to, but eventually new features will be added to the new format, which you won’t be able to benefit from if you aren’t using it.

As I mentioned in the previous post, internationalization is now open again, so if you’re interested in doing a translation, contact me.

As always, downloads are available on the GitPHP page, and bugs / enhancement requests can be reported on the Mantis bugtracker – with ~12000 new lines of code, I’m certaintly expecting a few unforseen bugs. Source code is always browsable on the live GitPHP.

You can always let me know what you think of the rewrite (or the project in general) by multiple means: email, commenting on my site, twitter, etc.

I hope you enjoy the new code.

With that said, now GitPHP is, in theory, able to work with any locale as long as someone translates the strings. The only thing I’ve done so far is string tokenization – I could probably pull out the date formats and allow those to vary by locale, however right now it’s not a big priority because the dates used throughout GitPHP are mostly ISO dates not specific to any locale. Also right now the language is defined at the config level – I intend to add the ability in the future for users to choose their own language.

So now, for the second time, a call… if you’re interested in translating GitPHP’s strings into another language, contact me at xiphux@gmail.com. I know some of you contacted me earlier the first time I made this call, but I decided to hold off since I knew this rewrite was coming. I apologize, but I’ve probably lost track of your emails – so if you’re still interested, please contact me again.