Like I did in Jan 2010, I wanted to reflect on some of the technologies I learned and things I did over the last 2 years…

Backend:

• Learned tons about scalability! Scaled Quizlet to serve 11 million visits per months (60 million pageviews, 5 million uniques), serving mostly UGC content with an average page response time of < 50ms.
• Got very comfortable with understanding and tweaking advanced configuration options of Apache, PHP, & MySQL for desired functionality and scalability requirements
• Went from a command line beginner to feeling very comfortable in Linux and Solaris, frequently writing bash/shell scripts for common sysadmin tasks, and trying to automate as much as possible.
• Worked with large data sets: our database had several hundred million rows and was in the dozens of gigabytes in size.
• Played a dev-ops role and helped transition all production servers to use Puppet for configuration management. Focused on having a process for all server related changes and created the ability to have over a dozen different servers recreate-able in very little time.
• Learned how to make MySQL fast on a high-trafficked and large database, do replication and have a quick recovery plan. Dealt with configuring databases, designing schemas, and optimizing SQL statements. Also managed our relationship with Percona’s consultants (leading MySQL experts) and learned from them.
• Automated backups and ensured we had quick access to them in the right places plus securely storing them off-site to Amazon S3 for safety. We had people relying on their data being accessible for studying and thankfully were able to recover quickly from incidents without losing any data.
• Sphinx Search is a very powerful open source search server good for indexing and quickly searching over MySQL. I learned how to configure it for our purposes and use its various features to power search on millions of rows of Quizlet content in a couple hundred milliseconds.
• Implemented and used several performance monitoring tools to keep tabs on a heavily trafficked website: Nagios, Munin, New Relic, etc.
• Learned all about (and implemented) email deliverability best practices: SPF records, DKIM, rDNS, Feedback Loops, IP address reputation, etc.
• Used several cloud service providers (Joyent, Slicehost, Amazon AWS). Setup reverse proxy CDNs for optimal performance of static files.

Frontend:

• Major focus on serious performance optimizations across entire stack. Spent time obsessing over how to get an extra 20ms out of a page load from our backend while keeping code clean. Resulted in an average page load of < 50ms across Quizlet.
• Also became extremely good at front-end performance; Did JavaScript profiling, proper handling of static files, caching headers, etc. Attended Velocity Conferences; Kept tabs on PageSpeed and WebpageTest waterfall results.
• Used Google Analytics Event Tracking for tracking client-side user events via JavaScript. Good for error detection and user behavior analysis.
• I already knew PHP well, but I’d say now I’m more of an expert at properly structuring object-oriented PHP5 on a large codebase.
• Learned all about and then implemented website localization & internationalization (i18n) for Quizlet, helping create a framework for user-generated translation submissions that launched in 7 languages.
• Got deep into Unicode and character encodings and implemented a language detection system in PHP based on n-grams and character encodings.
• Spent time optimizing with AdSense - A/B tested with different ad placement, targeting, and IAB ad sizes for better inventory.
• Wrote a very fast A/B testing framework and used it to optimize our upgrade page for better monetization
• Went very deep on the MooTools JavaScript library using it to build complex study games. In the past, jQuery has been my JS library of choice, but MooTools is very nice at being object-oriented and not afraid to extend native objects.
• Used Google Maps & Geocoding  API heavily (JavaScript and PHP) to build an events directory on CollectorsWeekly.
• Used the Stripe API to accept credit card payments (replacing our previous PayPal solution).
• Lots of cross-browser CSS – making things look good in browsers both old and new.

Dev Processes:

• Helped lead the development team with best practices and good processes for coding and communication.
• Became really good at Git and transitioned our team from SVN/Trac to Git/GitHub. Used git submodules to manage many of our dependancies and became a better open source citizen by submitting pull requests and bug reports in more projects.
• Did a lot of code reviewing on trac and GitHub with peers and new hires / interns.
• Introduced unit testing in both PHP (via PHPUnit) and JavaScript (via Jasmine); did some test-driven-development (TDD).
• Created a safer deployment process which first runs unit tests, PHP Lint, and other sanity checks before deploying.
• Started daily standups with the eng team to keep everyone up-to-date and on track.

Product:

• Along with development work, I was doing a lot of Product Management: feature triage and prioritization, tracking roadmaps, deciding among tradeoffs, etc.
• Huge focus on the user interface: tried to make every new feature high quality and easy to use. Spent a lot of time redoing the user flow of features that already existed to improve their ease of use. Removed clutter and made features that “just work”.
• Used Photoshop to work on visual improvements — designing or redesigning almost all the main pages and many shared UI components on Quizlet.
• Brainstormed and implemented game mechanics and other ways to make studying on Quizlet more fun and social.

Business+Management

• Focused heavily on recruiting other developers, helped create a process for finding potential candidates and putting them through a funnel. Did a lot of technical interviewing by phone/Skype and in-person. Helped build a great team of full-time people and interns.
• Team building. Helped manage and guide new employees, getting them up to speed and supporting their projects.
• Did a fair amount of biz dev as well. Negotiated deals for licensing software, worked out details of contracts. Talked with potential partners.
• Managed projects that other developers were working on, pushing for both code quality, fast completion, and a high quality user experience.
• Wrote a bunch of Quizlet blog posts about product announcements.

I’m sure I’m missing lots of stuff, and I should probably take even more time to write down some higher level lessons learned about startups. But for now this will do: startups are awesome and the best way to learn a lot of stuff really quickly by doing it. And if you’re lucky like I’ve been, it will be with an awesome team working on a great product used by tons of people.

I’ve recently moved onto a new opportunity that I’m very excited about. But I’m really glad to have been at Quizlet the last two years and I look forward to seeing it continuing to succeed widely!

(P.S., they’re hiring…)

I’ve been working with the founders, Dave Margulius and Andrew Sutherland (student at MIT), for around 6 months remotely and on short term trips as a contractor, and have now decided to accept an offer with the team full-time.  I’ll be responsible for developing new features on both websites, improving user experience, increasing monetization, and overall growing the businesses.

Quizlet.com is a studying website that lets students make flashcards online and study in a more effective and more fun way.  The site serves over a million students each month, has over 750,000 registered users, and has over 50 million user-generated flash cards.  As only the second developer on Quizlet, I’m excited to grow the website in huge ways and spend most of my time helping make studying better for a lot of people.

Collectors Weekly is a resource for all things collecting, antiques, and vintage.  It’s a combination of great original content and providing tools to browse eBay in a better way (SuperBrowse example).  I just helped launched its new geo-targetted events section and will spend about one third of my time developing new features for this site.

Both websites run on the LAMP stack.  I’ll be working heavily with Apache, PHP, JavaScript / Ajax, MySQL, Memcache, Sphinx, etc. to help grow traffic and features in a scalable way.  A decent number of the interesting technologies that I used for the first time in 2009 were a result of working with these guys.

This is an exciting time and a big opportunity for me as I’ll be able to significantly impact the growth and success of the startup.  It will also nice to be able to focus all my work attention in one direction rather than juggling many projects, as I’ve just graduated from UF and won’t be continuing freelance web work.  And of course, moving to San Francisco from Florida will be a big change and I’m excited, although I’ll still get to fly back every so often.  Wish me luck!

Chapter 1: Introduction & Overview, Chapter 2: Quick Tricks – Good chapter to teach beginners to both JavaScript, jQuery, and PHP the differences between client-side and server-side code, and how the jQuery framework fits into it all.

Chapter 3: Tabs and Accordions – At first this chapter looks to be nothing more than glorified examples of jQuery UI Widgets, but later it actually does a nice job showing how PHP can be used with these widgets to do server-side management of the tabs.

Chapter 4: Forms and Form Validation – Interesting and insightful way of validating forms (both on the client and server side, which is necessary) without having to duplicate all the validation rules.

Chapter 5: File Management, Chapter 6: Calendars – Very specific examples, but Ch. 6 uses the very nice jquery-week-calendar plugin to create an interface very similar to Google Calendar and shows how to link it up with PHP managing the events server-side.

Chapter 7: Image Manipulation – Another nice example of bringing using both PHP and jQuery plugins (treeview and jcrop) to create something really useful.  In this case an image browser, resizer, rotator, and cropper.

Chapter 8: Drag and Drop – Standard sorting/dragging examples + persisting data.

Chapter 9: Data Tables – Using the DataTables jQuery plugin with PHP on very large data sets.

Chapter 10: Optimization – A few helpful tips regarding caching, automatically merging .js source files, JavaScript code optimization, perceived user load times, etc.

Overall, if you’re a programmer not super familiar with PHP or JavaScript, this book does have some very helpful explanations with good examples.

One small gripe about the book: while I do know that black and white books are much cheaper to produce, reading code without syntax coloring makes it unnecessarily harder to understand.

To learn more or purchase: “jQuery 1.3 with PHP” by Kae Verens.

[Jump to Download/Installation Link]

Each time you set up a website with ExpressionEngine you probably want to install several different modules, extensions, and plugins. Doing that usually involves downloading a .zip file for each add-on, then individually unzipping and uploading each file/folder to the corresponding modules/, extensions/, plugins/, language/, and themes/ folders. Then each time any add-on is updated, you have to repeat the process across all your websites.

With my module, installing other add-ons is basically a one-click process. You just visit the module page (shown below) and then either upload a .zip/.php file, or give it the URL to the source from the developer’s website. Hit one button and my module takes the source file, extracts, and searches through the files and determines where each file/folder goes.

Screenshot:

For example, if you just upload a .zip file that contains a folder for a module, a pi.someplugin.php, and a lang.someplugin.php, all of the files will get extracted and moved to their proper locations. If it comes across a file it doesn’t understand, it will let you know.

Requirements:

• ExpressionEngine 1.6.8
• PHP cannot be in Safe Mode
• You may need to change permissions for a few folders if instructed

These likely would only be a problem with some shared hosting companies. (If you need good ExpressionEngine hosting let me know).

Installation:

1. Download freo_addon_installer_1.2.zip and unzip
2. Upload the freo_addon_installer/ folder to your system/modules/ folder
3. From the Modules page of your EE Control Panel, click “Install” next to “Freo Add-on Installer”
4. If you’re running a version of EE older than 1.6.8, copy freo_addon_installer/language/english/lang.freo_addon_installer.php to system/languages/english/
5. Please donate to support development of this and future add-ons.

Changes:

• v1.2 – Bug fixes (Oct. 25, 2009)
• v1.1 – Fixed problem when URLs redirect, like on github download zip URLs.  Added better support for installing fieldtypes for Brandon Kelly’s FieldFrame extension.  (Oct. 7, 2009)
• v1.0 – Initial Release. (Oct. 7, 2009)

Ideas & Plans:

• Create a long list of known add-ons to choose from so that installing will truly be one-click and not require looking up a source .zip URL.
• Integrate with LG Addon Updater so installing updates will be more of a breeze.

Feedback & Support:

If you encounter any problems or have any ideas for improvement at all please let me know. Leave a comment below and let me know how you like it. This support thread in the forums is the best place to resolve any problems you may have.

Please donate to support development of this and future add-ons.

I’ve been working with Magento for over a year now, and overall I’m extremely impressed.  The architecture of the system is certainly very complex, but once you start to poke around and understand why there are so many different directories and files, and how everything is organized, you’ll love the flexibility that it creates.

The biggest complaint of Magento is the lack of documentation. From the wiki pages, to user guides, to source code comments, the documentation is severely lacking and often out-of-date.  I hope this improves over time (I think it will).  What I’ve found is that you can eventually figure things out, but the learning curve is just a little steeper than it needs to be.  It is pretty necessary to understand many core concepts of Magento (how XML layout files are used, how data is stored, how template blocks and functionality interact, etc.) in order to correctly make even the simplest of changes.  But once you get it, you can do a lot.  I think the CMS ExpressionEngine does extremely well at this, in comparison, and they are both products aimed at similar audiences.

When I started developing on top of the Magento platform, I kept a log of some of the help things that I learned along the way, that I wish I had known upfront.  Many of these tips seem obvious upon looking back, but I thought I’d share them for anyone just getting started programming or designing for Magento.

For everybody:

• Use the Wiki, Knowledge Base, and forums
• Don’t ever modify core Magento files! Seriously.  This quick article is a must read for all beginners.
• To upgrade to a new version of Magento: just upgrade all of the components by using Magento Connect Manager.  This is found in the admin’s System menu, or by adding /downloader to the Magento home page URL.
• To add Google Analytics: simply add your Analytics account profile’s tracking number under Config > Google API
• To setup PayPal IPN: there is no need to change IPN settings in your PayPal account.  Magento automatically passes the IPN path with each checkout.
• Refresh the cache! If you make a change and wonder why it’s not showing up, use the dropdown on System > Cache Management.  It’s helpful to disable the cache during development.
• Learn how to customize via XML
• After installing Magento with sample data and SEO URL rewrites, the same data shows up on the index page but many of the links are broken, even with my .htaccess file present.  Solution: Refresh catalog rewrites in Admin > System > Cache > Refresh catalog rewrites
• Learn how to optimize and speed up Magento’s performance. It’s very slow by default but can be drastically improved. (I’ll be writing another blog post on this soon, but for now check out this thread.)

Design/theming tips:

• Read the Design Guide
• To figure out which template file to edit: turn on “Template Path Hints” through System > Configuration > Advanced > Developer > Debug > Template Path Hints = ON.  This option is only visible if you change the “Current Configuration Scope” dropdown to a website (like “Main Website”) rather than “Default Config”.
• To switch to a theme (such as “Modern”), after downloading it in Magento Connect Manager, go to System > Design and then click “Add Design Change”.

Programming tips:

• Understand the architecture
• To debug and output PHP variables: set System > Configuration > Advanced > Developer > Log Settings- > Enabled = Yes.  Then, from a CLI in your web root do “cd var/log/” and “tail -f system.log”.  In PHP, you can say Mage::log(“hello world”); to write to this log.
• Locate the PHP error log when getting a blank/white page, or an error message.  Mine was in public_html/error_log.

That’s all for now.  If you’re looking for a Magento designer or developer, get in contact with me.

By the way, the thing I hate about PHP most is that, because it’s so easy to get started with, you can’t immediately differentiate between people who really know PHP (with a solid understanding of security issues, scalability, MVC, object-oriented programming, design patterns, etc.) and people who know just enough to be dangerous.  I’ve seen way too many websites written with PHP and HTML scattered together, SQL injection vulnerabilities, lack of code reusability, etc. This problem isn’t specific to PHP only, but I find the percentage of novice PHP developers to be far worse than, say, Ruby or Python developers.

So I looked through the “php|architect’s Zend PHP 5 Certification” study guide for the last couple days and took an online practice test.  I actually learned a lot of interesting things about PHP that I didn’t know – so it was a good experience.

Today I took the exam at a local testing center and am happy to announce that I am officially a PHP 5 Zend Certified Engineer (ZCE).

By the way, if you’re looking to hire a good PHP developer, let me know (resume is here).  I graduate in December 2009.