Today I've been to the 5th Yandex Saturday event - Subbotnik. This is series of technical miniconferences in different cities. Place where yandexoids show other developers how they work, show toolset and other yandexish things. This was second Subbotnik for me. Last year most of the talks were about BEM (their front-end architecture) and toolset around it. So I've lost what they're talking about right from the second talk that day.
Today's Subbotnik was prepared much more thoughtfully. Yes, BEM was mentioned couple of times during each talk anyway, but all the themes were aimed to more broad audience, that doesn't know all of the ins and outs of bem-tools.
This Subbotnik had 2 tracks. One for Yandex's API users, the other was more for general developers with talks about front-end development and cloud services. I watched the second track, as I don't use Yandex API at all, so have no interest in it.
All the talks were in Russian most of the slides (if not all) too. But anyway I think it worth mentioning them in English speaking internet. You never know what may become useful.
Yandex.Disk. The flight is normal by Vladimir Rusinov.
Short introduction talk about Yandex.Disk from it's product manager. Yandex.Disk, for those who don't know, is online backup, storage and syncing service. Something very close to Dropbox. The product is doing fine, usage is growing, other services actively integrate with it using API. There were some numbers, history and funny picture. As it turned out it was warmup before second talk.
Operation and development of fast-growing clouds by Michail Belov and Oleg Leksunin.
First half of the talk was done by Oleg Leksunin who is system administrator of Yandex.Disk. He talked about requirements that were crucial for such service. architecture of the service, how they are organized and deployed. Also mentioned some key parts of the system and talked what each of them does.
There's so called MPFS service, that is a virtual file system for users's files. When file is added to YDisk request to MPFS is sent, which synchronously stores file as blobs on two different servers (with internal project Mulca). When it is saved file's metadata are saved to MongoDB.
Protocol for YDisk is extended WebDAV, that was chosen because it implements most of needed features, had existing working tools like wget or curl. And almost any modern OS has builtin WebDAV support. WebDAV server is written in Erlang. It was described in the post on habrahabr (in russian). MPFS is written in Python with Flask, Jinja2 and is served via nginx. Client applications use XMPP for push notifications.
Overall it was great talk with lots of interesting details and insights. Usually there's not so much you can take from talks like this, but it is interesting to know how big companies make big services. As it turns out everything may be quite simple.
Cryochamber for statics by Alexey Androsov I think was the best talk of the day. Alexey works as a fronted engineer at Yandex.Mail team. He started discussing how usually static files are served. Common case is to set cache period for static files for a long period, e.g. a year or a month, and make url to static resource uniqe by adding version of the app or revision (/static/1.1/app.js or /static/app.js?rev=a7cb9). Problem appears when there's only small change in the fronend app, then version or revision changes and all the static files have to be downloaded again. For big frontend apps this may become big problem, Y.Mail's static resource is more than 1Mb so with often changes to code there's a lot of things to download every time.
Yandex.Mail team took different approach. Now they put all the files for all of the versions into the same folder, but name of the resource is generated by file's hash. And postprocessor substitutes all links to the file in content that is served to user to hash-based new filenames.
So the CSS-code:
.class {
background: url("1.png");
}
.class {
background: url("freeze/f80abidmv6dlfo9.png");
}
Michail Korepanov with a talk on Incremental updates on the client continued discussion on how to make updates of front-apps less problematic. They decided to make browser download not the whole file with new version of app fiiles, but implement diff update.
So now the app doesn't load css and main javascript app as links, but they are downloaded via some javascript bootstrapper and saved to localstorage. After saving resources CSS is inserted inline to page and Javascript is evaluated with "new Function()".
Next time you open Y.Mail bootstrapper sends request with app version saved in localstorage, server responds with diff, bootstrapper patches code in localstorage and injects into page.
Diffs are prepared beforehand for 3 previous versions, when new version is deployed to production. If version stored in localstorage is too old, then server responds with whole latest version of the app. As I said, Yandex.Mail deploy new frontend app version twice a week, so such method saves a lot of bandwith and makes loading new version of the app much faster.
Also Michail mentioned RFC 3229 Delta encoding in HTTP, now finding diff and patching must be done by hand, with JavaScript, but this RFC implementetion could automate most this work. Unfortunately it is not implemented in any browser yet. There's a Chromium-based Yandex.Browser, so maybe this would be a place where this RFC would be first implemented ;)
Now I'm not sure how this is correlated to the previous talk by Alexey, since they solve the same problem from different angles. This talk looked a bit more theoretical, though it is used in production, as I understood.
Next talk after a short break - single-page application using node.js and BEM started by Eugene Filatov.
The main point of the talks was to tell about how yandex makes web apps that may be rendered both by backend and in the browser. Server rendering is needed for search engines to get your content properly.
BEM methodology operates with reusable blocks: news block, search bar block, email list block etc. Every block contains all in one place: html, css and javascript for it to work properly. But since block may be rendered both on server and on client there's some things that may need different handling. In the talk example of setting a cookie was mentioned. On server we have to add Set-Cookie request header, in browser we may write to document.cookies in the simplest case. But there's also a lot of shared code, so they prepare 3 files:
- cookie.js - code specific to the browser
- cookie.priv.js - code specific to node.js ("priv" is for "private")
- cookie.common.js - shared code for both versions
Eugene shared a piechart that shows that common code usually takes up to 54% of block code, so it definately worth having similar code organization if you want to do both server and browser rendering.
Next talk was Interface development in distributed teams by Sergey Puzankov.
That was just an overview how the whole process is organized, how communication is made and all the things. Not so much to tell about, everything is the same as others do. Well, there's two things that everybody should, but mostly don't have: regular inner conferences for sharing knowledge and up-to-date docs about systems.
Last talk was Code management or Why SCM by Sergey Sergeev.
Sergey is well-known git advocate, so he told about how his team uses git, best practices and anti-patterns. Now I know that Yandex SERP frontend team uses git-flow :) There was nothing really new here too, but it was nice opportunity to share user stories and ask questions for some people.
This was interesting event with great speakers. Despite there were no groundbreaking revalations, it was great opportunity to gather developers and discuss problems that all of us, both small dev teams and large software giants share.
Thanks Yandex Team for interesting event and great speakers!
