10. ā D O N A L D K N U T H ā
ā S T R U C T U R E D P R O G R A M M I N G W I T H G O T O S TAT E M E N T S ā
[ā¦] premature optimization is the root of all evil.
11. ā D O N A L D K N U T H ā
ā S T R U C T U R E D P R O G R A M M I N G W I T H G O T O S TAT E M E N T S ā
[ā¦] premature optimization is the root of all evil.
K E E P I N
M I N D
12. M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E
M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E
M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E
M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E
M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E
M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E
M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E
M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E
M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E
M E A S U R E
M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E
M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E
M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E
M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E
M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E
M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E
M E A S U R E M E A S U R E M E A S U R E M E A S U R E M E A S U R E
16. A PA C H E B E N C H M A R K
ab -n 200 -c 50 -k -g bench.tsv http://my.api.test/v1/pro...
^[1] ^[2] ^[3] ^[4] ^[5]
1. Number of requests
2. Concurrent requests
3. Use the keep-alive connection
4. Write the output ready for gnuplot
5. The URL to call
17. A PA C H E B E N C H M A R K
## Graph configuration ##
# Set the output format and size
set terminal jpeg size 1280,800
# Set the aspect ratio
set size 1, 1
# Title
set title "Benchmark testing"
# The legend/key position
set key left top
# Draw gridlines on the y axis
set grid y
# Label the x-axis
set xlabel 'seconds'
# Label the y-axis
set ylabel "response time (ms)"
## I/O Configuration
# Specify that the x-series data is time data
set xdata time
# The output file
set output "bench.jpg"
# Specify the *input* format of the time data
set timefmt "%s"
# Specify the *output* format for the x labels
set format x "%S"
# Use tabs as the delimiter instead of spaces
set datafile separator 't'
## Run the Plot
# Plot the data
plot "bench.tsv" every ::2 using 2:5 with points
exit
35. P L AY W I T H H E A D E R S
public function getAction($userIdentifier, Request $request)
{
$response = new Response();
$user = $this->fetchUser($userIdentifier);
$response->setETag($user->caclulateETag());
// $response->setLastModified($user->getLastUpdateDate());
if ($response->isNotModified($request)) {
return $response;
}
...
}
T O O M U C H
M A G I C I N S I D E
38. HTTP Compression is a set of rules
defined by the protocol which enable network
entities (Client and Server) to exchange
compressed data.
39. # Apache Configuration File
# ------------------------------------------------------------------------------
# | Compression |
# ------------------------------------------------------------------------------
<IfModule mod_deflate.c>
<IfModule mod_filter.c>
AddOutputFilterByType DEFLATE application/javascript
application/json
text/css
text/html
text/plain
text/xml
</IfModule>
</IfModule>
E N A B L E C O M P R E S S I O N
40. # Apache Configuration File
# ------------------------------------------------------------------------------
# | Compression |
# ------------------------------------------------------------------------------
<IfModule mod_deflate.c>
<IfModule mod_filter.c>
DeflateFilterNote Input input_info
DeflateFilterNote Output output_info
DeflateFilterNote Ratio ratio_info
LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate
CustomLog /var/log/apache2/deflate_log deflate
</IfModule>
</IfModule>
L O G C O M P R E S S I O N
41. H T T P C O M P R E S S I O N
GET /users/toretto460 HTTP/1.1
Host: www.example.com
Accept-Encoding: gzip, deflate
HTTP/1.1 200 OK
Date: mon, 27 Jul 2015 22:38:34 GMT
Server: Apache/1.3.3.7 (Unix) (Red-Hat/Linux)
Last-Modified: Wed, 08 Jan 2003 23:11:55 GMT
Accept-Ranges: bytes
Content-Length: 438
Connection: close
Content-Type: text/html; charset=UTF-8
Content-Encoding: gzip
42. H T T P C O M P R E S S I O N
GET /users/toretto460 HTTP/1.1
Host: www.example.com
HTTP/1.1 200 OK
Date: mon, 27 Jul 2015 22:38:34 GMT
Server: Apache/1.3.3.7 (Unix) (Red-Hat/Linux)
Last-Modified: Wed, 08 Jan 2003 23:11:55 GMT
Accept-Ranges: bytes
Content-Length: 438
Connection: close
Content-Type: text/html; charset=UTF-8
Accept-Encoding: gzip, deflate
Content-Encoding: gzip
43. T H E P O W E R O F H T T P
Client ServerISP Proxy
44. T H E P O W E R O F H T T P
Client ServerISP Proxy
Client cache
45. T H E P O W E R O F H T T P
Client ServerISP Proxy
Client cache
Public cache
46. T H E P O W E R O F H T T P
Client ServerISP Proxy
Client cache
Public cache
Reverse Proxy
Data compression
47. A G E N D A
ā¢ Performance
ā¢ HTTP
ā¢ DB Performance
ā¢ Heavy load tasks
49. T I P S
ā¢ AUTO_INCREMENT / SEQUENCE are not so cheap
ā¢ Understanding the TRACKING POLICY
ā¢ Use READONLY entities
ā¢ Letās CACHE
50. A U T O _ I N C R E M E N T & S E Q U E N C E S
A R E N O T S O C H E A P
class User
{
/**
* @ORMId
* @ORMColumn(name="ID", type="string", length=37, nullable=false)
* @ORMGeneratedValue(strategy="AUTO")
*/
private $id;
}
51. A U T O _ I N C R E M E N T & S E Q U E N C E S
A R E N O T S O C H E A P
class User
{
/**
* @ORMId
* @ORMColumn(name="ID", type="string", length=37, nullable=false)
* @ORMGeneratedValue(strategy="AUTO")
*/
private $id;
public function __construct(UserInfoDTO $userInfo)
{
$this->id = (string) Uuid::uuid4();
$this->name = $userInfo->name;
...
}
}
52. U N D E R S TA N D I N G T H E
T R A C K I N G P O L I C Y
53. U N D E R S TA N D I N G T H E
T R A C K I N G P O L I C Y
ā M A R T I N F O W L E R ā
āMaintains a list of objects affected by a business
transaction
and coordinates the writing out of changes and the
resolution of concurrency problems.ā
UNIT OF WORK
59. D E F E R R E D I M P L I C I T
With this policy, Doctrine detects the changes by a
property-by-property comparison at commit time and
also detects changes to entities or new entities that are
referenced by other managed entities ā¦
http://doctrine-orm.readthedocs.org/en/latest/reference/change-tracking-policies.html
60. D E F E R R E D I M P L I C I T
http://doctrine-orm.readthedocs.org/en/latest/reference/change-tracking-policies.html
class User
{
/**
* @ORMId
* @ORMColumn(name="ID", type="string", length=37, nullable=false)
*/
private $id;
/**
* @ORMColumn(name="FIRST_NAME", type="string", length=100, nullable=false)
*/
private $firstName;
/**
* @ORMColumn(name="LAST_NAME", type="string", length=100, nullable=false)
*/
private $lastName;
/**
* @ORMColumn(name="BIRTH_DATE", type="date", nullable=false)
*/
private $birthDate;
}
UOW
$em->flush();
61. D E F E R R E D E X P L I C I T
ā¦ the difference is that Doctrine 2 only considers
entities that have been explicitly marked for change
detection through a call to
EntityManager::persist(entity)
or through a save cascade.
http://doctrine-orm.readthedocs.org/en/latest/reference/change-tracking-policies.html
62. D E F E R R E D E X P L I C I T
http://doctrine-orm.readthedocs.org/en/latest/reference/change-tracking-policies.html
class User
{
/**
* @ORMId
* @ORMColumn(name="ID", type="string", length=37, nullable=false)
*/
private $id;
/**
* @ORMColumn(name="FIRST_NAME", type="string", length=100, nullable=false)
*/
private $firstName;
/**
* @ORMColumn(name="LAST_NAME", type="string", length=100, nullable=false)
*/
private $lastName;
/**
* @ORMColumn(name="BIRTH_DATE", type="date", nullable=false)
*/
private $birthDate;
}
UOW
$em->persist($user);
$em->flush();
63. N O T I F Y
This policy is based on the assumption that the entities
notify interested listeners of changes to their properties.
For that purpose, a class that wants to use this policy
needs to implement the NotifyPropertyChanged
interface
http://doctrine-orm.readthedocs.org/en/latest/reference/change-tracking-policies.html
64. N O T I F Y
<?php
use DoctrineCommonNotifyPropertyChanged,
DoctrineCommonPropertyChangedListener;
/**
* @Entity
* @ChangeTrackingPolicy("NOTIFY")
*/
class User implements NotifyPropertyChanged
{
// ...
private $_listeners = array();
public function addPropertyChangedListener(PropertyChangedListener $listener)
{
$this->_listeners[] = $listener;
}
}
65. N O T I F Y
<?php
// . . .
protected function _onPropertyChanged($propName, $oldValue, $newValue)
{
if ($this->_listeners) {
foreach ($this->_listeners as $listener) {
$listener->propertyChanged($this, $propName, $oldValue, $newValue);
}
}
}
public function setData($data)
{
if ($data != $this->data) {
$this->_onPropertyChanged('data', $this->data, $data);
$this->data = $data;
}
}
}
66. U S E R E A D O N LY E N T I T I E S
This means that the entity marked as read only is never
considered for updates
67. U S E R E A D O N LY E N T I T I E S
/**
* @ORMEntity(readOnly=true)
*/
class NewsTag
{
/**
* @ORMId
* @ORMColumn(name="ID", type="string", length=37, nullable=false)
*/
private $id;
}
68. L E T ā S C A C H E
/**
* Configure the Doctrine ORM with a result cache provider.
*/
$config = new DoctrineORMConfiguration();
// The APC way
$apcCacheDriver = new DoctrineCommonCacheApcCache();
$config->setResultCacheImpl($apcCacheDriver);
// The memcache way
$memcache = new Memcache();
$memcache->connect('memcache_host', 11211);
$memCacheDriver = new DoctrineCommonCacheMemcacheCache();
$memCacheDriver->setMemcache($memcache);
$config->setResultCacheImpl($memCacheDriver);
69. L E T ā S C A C H E
doctrine:
orm:
metadata_cache_driver: apc
query_cache_driver: apc
result_cache_driver:
type: memcache
host: localhost
port: 11211
70. L E T ā S C A C H E
class UserRepository
{
/**
* @return User[]
*/
protected function findLocked()
{
$qb = $this->createQueryBuilder('user');
$qb->where($qb->expr()->eq('user.locked', true));
$query = $qb->getQuery();
$query->useResultCache(true, 60 * 2 /* 2 minutes */);
return $query->getResult();
}
}
71. R E A L U S E C A S E - # 1 D ATA I M P O R T
> 2 0 0 K E N T I T I E S
SEQUENCE UUID
$em->flush(get_class($entity));
$em->clear(get_class($entity));
72. R E A L U S E C A S E - # 1 D ATA I M P O R T
> 2 0 0 K E N T I T I E S
73. R E A L U S E C A S E - # 1 D ATA I M P O R T
> 2 0 0 K E N T I T I E S
208k Entities
loaded in 2h 25m
š
74. R E A L U S E C A S E - # 1 D ATA I M P O R T
> 2 0 0 K E N T I T I E S
75. R E A L U S E C A S E - # 1 D ATA I M P O R T
> 2 0 0 K E N T I T I E S
AFTER THE CURE
208k Entities
loaded in 28 minutes
š
76. A G E N D A
ā¢ Performance
ā¢ HTTP
ā¢ DB Performance
ā¢ Heavy load tasks
77. S L O W TA S K
The user avatar should be resized to 300x300px
The administrator should upload up to 250k
orders in a single cvs file
Send an email when a new post has been created
78. S L O W TA S K
A crawler should find and aggregate product info
Every X minutes the rss feed should be updated
The user can export all the orders
80. R E A L U S E C A S E - # 2 R E P O R T I N G
AS AN analyst
I WANT TO request for a new monthly report
81. R E A L U S E C A S E - # 2 R E P O R T I N G
AS AN analyst
I WANT TO request for a new monthly report
CREATE TABLE TMP_CFMS_R7_7844 AS SELECT ROWNUM AS ID, x.* FROM (SELECT COD_DEALER, COD_PDV, TOTAL_ACQUISIZIONI,
CONFRONTO_ACQUISIZIONI, AVG_ACQUISIZIONI, PERCENTUALE_INCREMENTO, CAST( NULL AS NUMBER ) QTA_SOGL, CAST( NULL AS
NUMBER ) QTA_SUPE_SOGL, TOTAL_SOSPENSIONI, SOSPENSIONI_CONFRONTO, AVG_SOSPENSIONI, PERCENTUALE_INCREMENTO_SOSP,
CAST( NULL AS NUMBER ) QTA_SOGL_SOSP, CAST( NULL AS NUMBER ) QTA_SUPE_SOGL_SOSP, PERCENTUALE_SOSPENSIONI_ACQU,
PERC_SOSP_ACQU_CONF, PERCENTUALE_SUPERAMENTO, D_RAG_SOC, D_IND, D_LOCALITA, D_CAP, D_PROVINCIA, D_PIVA, INDIRIZZO
V_IND, LOCALITA V_LOCALITA, CAP V_CAP, PROVINCIA V_PROVINCIA FROM ANAGRAFICA_DEALER, (SELECT COD_DEALER, COD_PDV,
TOTAL_ACQUISIZIONI, CONFRONTO_ACQUISIZIONI, AVG_ACQUISIZIONI, PERCENTUALE_INCREMENTO, CAST( NULL AS NUMBER )
QTA_SOGL, CAST( NULL AS NUMBER ) QTA_SUPE_SOGL, TOTAL_SOSPENSIONI, SOSPENSIONI_CONFRONTO, AVG_SOSPENSIONI,
PERCENTUALE_INCREMENTO_SOSP, CAST( NULL AS NUMBER ) QTA_SOGL_SOSP, CAST( NULL AS NUMBER ) QTA_SUPE_SOGL_SOSP,
PERCENTUALE_SOSPENSIONI_ACQU, PERC_SOSP_ACQU_CONF, PERCENTUALE_SUPERAMENTO, RAGSOC D_RAG_SOC, INDIRIZZO D_IND,
LOCALITA D_LOCALITA, CAP D_CAP, PROVINCIA D_PROVINCIA, PIVA D_PIVA FROM ANAGRAFICA_DEALER, (SELECT
A.COD_DEALER,A.COD_PDV,NVL(A.TOTAL_ACQUISIZIONI,'0') TOTAL_ACQUISIZIONI,NVL(A.CONFRONTO_ACQUISIZIONI,'0')
CONFRONTO_ACQUISIZIONI,NVL(B.AVG_ACQUISIZIONI,'0') AVG_ACQUISIZIONI,NVL(B.PERCENTUALE_INCREMENTO,'0')
PERCENTUALE_INCREMENTO, CAST( NULL AS NUMBER ) QTA_SOGL, CAST( NULL AS NUMBER )
QTA_SUPE_SOGL ,NVL(C.TOTAL_SOSPENSIONI,'0') TOTAL_SOSPENSIONI,NVL(C.SOSPENSIONI_CONFRONTO,'0')
SOSPENSIONI_CONFRONTO, NVL(D.AVG_SOSPENSIONI,'0') AVG_SOSPENSIONI,NVL(D.PERCENTUALE_INCREMENTO_SOSP,'0')
PERCENTUALE_INCREMENTO_SOSP, CAST( NULL AS NUMBER ) QTA_SOGL_SOSP, CAST( NULL AS NUMBER )
QTA_SUPE_SOGL_SOSP ,NVL(E.PERCENTUALE_SOSPENSIONI_ACQU,'0') PERCENTUALE_SOSPENSIONI_ACQU,
NVL(PERC_SOSP_ACQU_CONF,'0') PERC_SOSP_ACQU_CONF,NVL(PERCENTUALE_SUPERAMENTO,'0') PERCENTUALE_SUPERAMENTO FROM
(SELECT ROWNUM AS ID, x.* FROM (SELECT COD_DEALER, COD_PDV, TOTAL_ACQUISIZIONI, CONFRONTO_ACQUISIZIONI, CAST( NULL
AS NUMBER ) QTA_SOGL, CAST( NULL AS NUMBER ) QTA_SUPE_SOGL, D_RAG_SOC, D_IND, D_LOCALITA, D_CAP, D_PROVINCIA,
D_PIVA, INDIRIZZO V_IND, LOCALITA V_LOCALITA, CAP V_CAP, PROVINCIA V_PROVINCIA FROM ANAGRAFICA_DEALER, (SELECT
COD_DEALER, COD_PDV, TOTAL_ACQUISIZIONI, CONFRONTO_ACQUISIZIONI, CAST( NULL AS NUMBER ) QTA_SOGL, CAST( NULL AS
NUMBER ) QTA_SUPE_SOGL, RAGSOC D_RAG_SOC, INDIRIZZO D_IND, LOCALITA D_LOCALITA, CAP D_CAP, PROVINCIA D_PROVINCIA,
PIVA D_PIVA FROM ANAGRAFICA_DEALER, ( SELECT
A.COD_DEALER,A.COD_PDV,A.TOTAL_ACQUISIZIONI,NVL(B.TOTAL_ACQUISIZIONI,'0') CONFRONTO_ACQUISIZIONI, CAST( NULL AS
NUMBER ) QTA_SOGL, CAST( NULL AS NUMBER ) QTA_SUPE_SOGL FROM (SELECT A.COD_DEALER,A.COD_PDV,COUNT(A.COD_DEALER)
TOTAL_ACQUISIZIONI, CAST( NULL AS NUMBER ) QTA_SOGL, CAST( NULL AS NUMBER ) QTA_SUPE_SOGL FROM CFMS_ATT A WHERE
A.DATA_ATTIVAZIONE between TO_DATE('2015-06-15 00:00:00','yyyy-mm-dd HH24:MI:SS') AND TO_DATE('2015-06-15
23:59:59','yyyy-mm-dd HH24:MI:SS')
AND A.COD_PDV IS NOT NULL GROUP BY A.COD_DEALER,A.COD_PDV) A LEFT JOIN ( SELECT
COD_DEALER,COD_PDV,COUNT(COD_DEALER) TOTAL_ACQUISIZIONI FROM CFMS_ATT WHERE DATA_ATTIVAZIONE between
TO_DATE('2015-06-16 00:00:00','yyyy-mm-dd HH24:MI:SS') AND TO_DATE('2015-06-16 23:59:59','yyyy-mm-dd HH24:MI:SS')
AND COD_PDV IS NOT NULL GROUP BY COD_DEALER,COD_PDV ) B ON A.COD_DEALER=B.COD_DEALER AND A.COD_PDV=B.COD_PDV) WHERE
COD_DEALER = COD_ID (+)) WHERE COD_PDV = COD_ID (+) ORDER BY COD_DEALER) x ) A FULL OUTER JOIN (SELECT ROWNUM AS
ID, x.* FROM (SELECT COD_DEALER, COD_PDV, ACQUISIZIONI, AVG_ACQUISIZIONI, ACQUISIZIONI_CONFRONTO,
AVG_ACQUISIZIONI_CONFRONTO, PERCENTUALE_INCREMENTO, D_RAG_SOC, D_IND, D_LOCALITA, D_CAP, D_PROVINCIA, D_PIVA,
INDIRIZZO V_IND, LOCALITA V_LOCALITA, CAP V_CAP, PROVINCIA V_PROVINCIA FROM ANAGRAFICA_DEALER, (SELECT COD_DEALER,
COD_PDV, ACQUISIZIONI, AVG_ACQUISIZIONI, ACQUISIZIONI_CONFRONTO, AVG_ACQUISIZIONI_CONFRONTO, PERCENTUALE_INCREMENTO,
RAGSOC D_RAG_SOC, INDIRIZZO D_IND, LOCALITA D_LOCALITA, CAP D_CAP, PROVINCIA D_PROVINCIA, PIVA D_PIVA FROM
ANAGRAFICA_DEALER, ( SELECT A.COD_DEALER,A.COD_PDV,A.ACQUISIZIONI,TO_CHAR(A.AVG_ACQUISIZIONI,'99990D99')
82. R E A L U S E C A S E - # 2 R E P O R T I N G
AS AN analyst
I WANT TO request for a new monthly report
83. R E A L U S E C A S E - # 2 R E P O R T I N G
AS AN analyst
I WANT TO request for a new monthly report
SO THAT the system will send me an email
with the attached csv report
85. S C A L I N G T H E A S Y N C
Y O U R A P P
I S
H E R E
W O R K E R
W O R K E R
W O R K E R
W O R K E R
Q U E U E
POST /rss
PATCH /users
POST /export
86. A S Y N C H R O N O U S TA S K
ā¢ https://github.com/uecode/qpush-bundle
ā¢ https://github.com/videlalvaro/php-amqplib
ā¢ https://github.com/videlalvaro/RabbitMqBundle
ā¢ http://gearmanbundle.readthedocs.org/en/latest/configuration.html
ā¢ https://github.com/chrisboulton/php-resque