Nerds who love the symfony-project
9 Dec
Darren Hoyt and I rolled out a new website yesterday. WP Questions is a place for WordPress developers to go when they have questions so urgent that they are willing to pay money to get an answer. We built the site using Symfony, and I’d like to use this site to talk about the situations where I think Symfony makes a great choice as a development platform.
First, understand our goals. We needed to create a site that allowed questions to be posted, money to be taken, answers to be given, and money dispersed to the person who had offered the most useful answer. Darren Hoyt did a good job of summarizing the goals of the site.
Right now we are listening to feedback and fine-tuning the site. Once we feel we have fixed all the bugs and added all the needed features, we plan to roll out an identical site for each of the major technologies: Java, Ruby, Rails, Symfony, Cake, Drupal, .NET Oracle, SQL, Javascript, etc.
So what do we need in our development platform?
1.) We can not guess the future of our data storage so we need database abstraction that keeps SQL out of our code.
2.) We know the structure of our data will evolve, so we need an ORM that can mask the changes.
3.) We know the designs will evolve and multiply even as the data schema evolves, so we need a clean MVC architecture that allows us to correctly observe a separation of concerns.
4.) We know that if we are successful we will eventually need to hire other programmers, so we need a well-documented framework that other programmers know, preferably one that is open source.
5.) We need to maintain the code base even as we scale up to running 100 sites, so we need a framework that allows us to run 100 sites off a single installation.
So what should we use?
A few months ago it was suggested that we use WordPress to develop the site. I think this is clearly a bad idea. WordPress has a fairly rigid database structure. A developer can add new database tables to any WordPress installations, but establishing relationships between the new tables and the core tables is always a bit of a hack. And WordPress engages in some practices that I regard as extremely questionable, such as storing config data in the database and using fields in the database to store formatted text that needs to be drawn out and parsed for data - data storage inside of data storage.
We could have also started with forum software like phpBB and then hacked it to add in the other features that we wanted. But again, it would be an uphill struggle against a database structure that makes too many assumptions about what we need, and which offers us few tools to evolve the structure over time. And, personally, I find their template system a pain to edit.
Last month I complained about the prominence of Drupal. The comments in that thread show that Drupal has some very strong supporters. However, Drupal started life as a CMS, and still shows some traces of that, in terms of assumptions about database structure. Also, its files live within the directory of the server that is made available to the public on the web, something I regard as a security risk. Karl made a good point in a comment in that thread:
Interesting here is that I looked at drupal, but choose not to use it, instead, Symfony was the choice. The two biggest reasons was 3rd party integration with quickbooks and scalability, we were afraid that drupals datamodel would cause us headaches at the end.
I like the way that James Herrmann summed things up at the end:
What are the questions asked that lead to the answer of ‘Symfony’, ‘Cake’, or ‘Solar’? I have a feeling they would entail infinite extensibility, unit testing, bridging with other libraries, optimization and security, to name a few.
Clearly, using a CMS is a bad idea when you know you’ll need a maximum of flexibility in the future. But why use Symfony? Why not use one of the other frameworks? Grails offers a clean architecture, a great plugin system, a highly productive dynamic environment. Groovy, and the JVM, is somewhat faster than PHP, so it offers a speed boost which might be important when we need to scale up. I’ve written enough Groovy code that I could probably write the app in Grails if I wanted to. So why not? My reasons here have to do with the overall eco-systems that surround the 2 languages. There is an abundant supply of PHP programmers. More importantly, the sysadmins I know are comfortable with Linux, Apache and PHP, but not with Tomcat. In the bad old days, the whole world of Java was geared toward Fortune 500 companies with big budgets. The frameworks were as verbose as the underlying language. The idea of dynamic languages and lightweight frameworks is relatively new to the world of the JVM. PHP, by contrast, has been stable for 10 years now, and it has a huge following.
There are, of course, a hundred other technologies I might have considered, but I am limited to the ones I know well.
Symfony is going to allow us to grow and evolve the database structure while keeping the code clean. I’ll give you an example. On our Top Winners page we offer a few ways of sorting. The page shows the users, their name, their image and how much they’ve won. I must draw the data out of these 2 tables:
sf_guard_user_profile:
_attributes: { phpName: sfGuardUserProfile }
id: ~
user_id: { type: integer, foreignTable: sf_guard_user, foreignReference: id, required: true }
status: { type: varchar(255), required: false }
first_name: { type: varchar(255), required: false }
last_name: { type: varchar(255), required: false }
email: { type: varchar(255), required: false }
url: { type: varchar(255), required: false }
biography: { type: longvarchar, required: false }
created_at: ~
updated_at: ~
cash_total: { type: float, required: false }
image: { type: VARCHAR, size:255 }
secret_signup_key: { type: VARCHAR, size:255 }answer:
id: ~
user_id: { type: integer, foreignTable: sf_guard_user, foreignReference: id, required: false }
question_id: { type: integer, foreignTable: question, foreignReference: id, required: false }
image: { type: varchar(255), required: false }
description: { type: longvarchar, required: false }
status: { type: varchar(255), required: false }
score: { type: integer, required: false }
prize_amount_paid_for_this_answer: { type: float, required: false }
created_at: ~
updated_at: ~
I need to create a JOIN. There is no foreign key relationships between these 2 tables, so the automatically generated Propel classes do not have any JOIN methods. I prefer to handle this by creating a database view,
CREATE VIEW answers_totals_users AS
SELECT answer.*,
COUNT(question_id) as questionTotal,
SUM(prize_amount_paid_for_this_answer) as prizeTotal,
sf_guard_user_profile.first_name, sf_guard_user_profile.last_name,
sf_guard_user_profile.image AS userImage
FROM answer, sf_guard_user_profile
WHERE answer.user_id=sf_guard_user_profile.user_id
GROUP BY answer.user_id
and then I add it to schema.yml as if it was a real database table:
answers_totals_users:
id: ~
user_id: { type: integer, foreignTable: sf_guard_user, foreignReference: id, required: false }
question_id: { type: integer, foreignTable: question, foreignReference: id, required: false }
image: { type: varchar(255), required: false }
description: { type: longvarchar, required: false }
status: { type: varchar(255), required: false }
score: { type: integer, required: false }
prize_amount_paid_for_this_answer: { type: float, required: false }
prizeTotal: { type: float, required: false }
questionTotal: { type: integer, required: false }
created_at: ~
updated_at: ~
first_name: { type: varchar(255), required: false }
last_name: { type: varchar(255), required: false }
userImage: { type: VARCHAR, size:255 }
Then I run the “symfony propel:build-model” command, and now I have a model class that selects against this JOIN as if it was a single table. This leaves my action code looking simple and clean:
$limit = $request->getParameter(’limit’);
if (!$limit) $limit = 25;
$c = new Criteria();
$c->setLimit($limit);
$c->add(AnswersTotalsUsersPeer::PRIZETOTAL, 0, Criteria::GREATER_THAN);
$c->addDescendingOrderByColumn(AnswersTotalsUsersPeer::PRIZETOTAL);
$this->listOfTopWinners = AnswersTotalsUsersPeer::doSelect($c);
I started to think of a site such as this one some years ago, when I first used Experts Exchange. I was frustrated that Experts Exchange took my money, yet none of the money went to the people who were helping me. I wanted a site where I could give money directly to the people answering my questions. My ideas about this site have become more clear over time, and no doubt 6 months from now I’ll have even more clarity about how this site/software can be useful to people. Likewise, as we get more feedback from our users we will continue to add new features, and offer more information. They’ll be a need for more JOINs and sometimes new database tables. A framework like Symfony assumes a changing database structure, it is set up to help a developer deal with those changes, and in this it is different from the CMSs that assume they know what kind of site you are building. Symfony, therefore, offers an attractive set of features for developing our site/software over the long term. It is the ideal development platform for us.
6 Responses for "A new website, and some thoughts on choosing Symfony"
You could even save code with that :
$request->getParameter('limit', 25)Best of all, queries like that should even be abstracted to the AnswersTotalsUsersPeer class, just to have a
AnswersTotalsUsersPeer::doSelectByHighestPrizeTotal($request->getParameter('limit', 25))Better action logic. Better reusability.
Symfony is great for sure and like you, I don’t think WP is flexible enough, with good performances, to do such things.
However, it’s quite strange you create a view, and add some logic in database against managing logic within the code. Which is easier to deploy and maintain.
That is a sweet idea you’ve got there, and you’ve laid out out a good decision making process. The view idea is brilliant! I’ll be using that, thanks.
Rock on.
Daniel
[...] used the Symfony framework to build this site. I’ve written a somewhat technical post, over on Symfony Nerds, explaining why this was a good choice for [...]
Oncle Tom, there are some very good reasons not to use database views on any project. I have some worries about performance, given the problems described in MySql forums and weblogs such as these:
http://forums.mysql.com/read.php?100,164811,164999#msg-164999
http://www.mysqlperformanceblog.com/2007/08/12/mysql-view-as-performance-troublemaker/
One can avoid those gotchas if one is careful. And, of course, I expect MySql to get better about handling views.
I take this approach, of using views, for 4 main reasons:
1.) Development time - it is quick to let “symfony propel:build-model” write all of my model code.
2.) Documentation - I’ll be handing this project over to other programmers. Rather than having a dozen custom JOIN methods scattered through a dozen model classes, future programmers can look in schema.yml and see every JOIN that exists in the application.
3.) Optimization - for reasons of performance, it’s important to build an index on every column that you use in a JOIN, or in a WHERE clause. In the past, I’ve found it difficult to discover which database table columns were used in WHERE clauses. That is, for years, I would write software, and then try to optimize database performance, which then entailed research into how the database is actually being used. This sometimes gets complicated on projects where multiple programmers were working. Keeping all such data in schema.yml makes things easy (you may regard this as a repeat of #2).
4.) Portability - repeating part of #2 - I understand what you are trying to say about portability. Your point is “What if we later move to a data storage technology that doesn’t support JOIN syntax?” For instance, what if in the future we decided we wanted to keep all of our data in a big XML flat file? XML files do not have VIEWS, so what are we going to do then? But I see this issue exactly the opposite way that you see it. Our software is expecting some kind of view that holds a particular kind of data. The schema.yml is the ideal place to record what kind of data the software is expecting, or rather, the views which the software assumes it can rely upon.
Suppose that 4 years from now the decision is made to move to a XML flat file. Suppose I’m no longer around, and different programmers are in charge of the transition. They should be able to take one glance at schema.yml and see exactly what views of data the software is expecting. Keeping this data all in schema.yml seems much more portable to me than having a dozen or so custom JOIN methods, in a dozen different models, which might need to be re-written.
Of course, no one is putting a gun to our head and telling us to move to a flat XML file. That is the worst case scenario, and I use it only to address your concerns. What is more likely is that, if the site scaled up and became huge, we might move to a RDBMS that had a more robust handling of VIEWs. For instance, maybe the decision would be made to move to Oracle. Then using a VIEW would offer a big performance advantage over using a JOIN.
You’ve probably heard the idea that JOINs Are Evil
If the site got really huge, then we would look for ways to de-normalize our data in ways that allowed for better performance. We might end up relying on temporary, pre-joined table for everything, possibly using an RDBMS that makes this easy (again, Oracle comes to mind). Or, maybe, we would look into the more radical approaches now being explored by technologies like CouchDb or Hadoop. However, this is not a problem that we would try to solve in a method in our model class. The problem eventually becomes bigger and more fundamental than anything you can address in your model class.
[...] Just this last summer I worked on a project with Scott Meves. We were using the Symfony framework, the same framework that WP Questions is built in. Scott is a true genius when it comes to Symfony. I recall we needed to do some JOIN calls to the [...]
[...] Just this last summer I worked on a project with Scott Meves. We were using the Symfony framework, the same framework that WP Questions is built in. Scott is a true genius when it comes to Symfony. I recall we needed to do some JOIN calls to the [...]
Leave a reply