Extracting Seasonality and Trend from Data: Decomposition Using R

Clique na imagem para seguir o link.

Uma excelente descrição da decomposição clássica com Python e R.

Time series decomposition works by splitting a time series into three components: seasonality, trends and random fluctiation. To show how this works, we will study the decompose( ) and STL( ) functions in the R language.

Understanding Decomposition

Decompose One Time Series into Multiple Series

Time series decomposition is a mathematical procedure which transforms a time series into multiple different time series. The original time series is often split into 3 component series:

  • Seasonal: Patterns that repeat with a fixed period of time. For example, a website might receive more visits during weekends; this would produce data with a seasonality of 7 days.
  • Trend: The underlying trend of the metrics. A website increasing in popularity should show a general trend that goes up.
  • Random: Also call “noise”, “irregular” or “remainder,” this is the residuals of the original time series after the seasonal and trend series are removed.

Tags: , , ,

Biased vs Unbiased: Debunking Statistical Myths

clique na imagem para seguir o link

Uma reflexão sobre os enviesamentos que usamos na ciência de dados.

Anyone who attended statistical training at the college level has been taught the four rules that you should always abide by, when developing statistical models and predictions:

  1. You should only use unbiased estimates
  2. You should use estimates that have minimum variance
  3. In any optimization problem (for instance to compute an estimate from a maximum likelihood function, or to detect the best, most predictive subset of variables), you should always shoot for a global optimum, not a local one.
  4. And if you violate any of the above three rules, at least you need to make sure that your estimate, when the number of observations is large, satisfies them.

As a data scientist and ex-statistician, I violate these rules (especially #1 – #3) almost daily. Indeed, that’s part of what makes data science different from statistical science.



Voronoi diagram from smooshing paint between glass

clicar na imagem para seguir o link

Uma abordagem original aos diagramas de Voronoi.

Tags: ,

The list of 2018 visualization lists

clique na imagem para seguir o link

Boa e longa lista de todo o tipo de visualizações.

The list of 2018 visualization lists
December 31, 2018
Officially a yearly habit now: the the list of visualization lists. So here is my list of visualisations, charts, graphics, maps, satellite journalism and science photography lists, version 2018.

Stories, Charts and Maps

@FlowingData: Best Data Visualization Projects of 2018

@ReutersGraphics: The Reuters graphics department takes a lookback at a year’s worth of work

@FiveThirtyEight: The 45 Best — And Weirdest — Charts We Made In 2018

@GuardianVisuals: 18 for 2018: a thread of our biggest projects of the year

@SCMPGraphics: 2018 in visuals: South China Morning Post’s infographic highlights

@qz: The best data visualization in 2018, according to data visualization experts

@HackAStory: The 40 best digital stories of 2018 listed for you by Hackastory

@EconDailyCharts: The 2018 Daily Chart advent calendar

@visualisingdata: 6 monthly reviews of the best of data visualisation

@ftdata: Charts of the Year 2018: our writers’ picks
@WSJGraphics: The Year in Graphics 2018



Making it easier to discover datasets

Clique na imagem para seguir o link

Clique na imagem para seguir o link

Novo recurso da google para identificar conjuntos de dados.

In today’s world, scientists in many disciplines and a growing number of journalists live and breathe data. There are many thousands of data repositories on the web, providing access to millions of datasets; and local and national governments around the world publish their data as well. To enable easy access to this data, we launched Dataset Search, so that scientists, data journalists, data geeks, or anyone else can find the data required for their work and their stories, or simply to satisfy their intellectual curiosity.


IFORS Developing Countries OR Resources Website

Clique na imagem para seguir o link

Clique na imagem para seguir o link

Artigos e software relacionado com Investigação Operacional

Click below on required topic headings to access papers or click here to access International Abstracts in OR


SNS Trabalhadores por Grupo Profissional

sem nome

Bons dados sobre o SNS

Apurar o nº de trabalhadores (empregos), por instituição e por grupo profissional, com contrato de trabalho ativo no mês de análise.

Número de trabalhadores (empregos) com contrato de trabalho ativo no mês em análise, por entidade e por mês, discriminado pelos grupos profissionais: Médicos (sem contabilizar Internos), Médicos Internos, Enfermeiros, Técnicos Superiores de Saúde, Técnicos Superiores de Diagnóstico e Terapêutica, Assistentes Técnicos, Assistentes Operacionais, Técnicos Superiores, Informáticos e Outros.

Nota: Os dados apresentados dizem respeito aos trabalhadores vinculados com contrato de trabalho às entidades do setor público administrativo (SPA) e entidades públicas empresarias (EPE) que se encontram sob a tutela do Ministério da Saúde, aos quais acrescem ainda os profissionais que exercem funções nos estabelecimentos hospitalares em regime de parceria público-privada integrados no Serviço Nacional de Saúde. (Ver anexo – Número de Profissionais nos Estabelecimentos Hospitalares em Regime de Parceira Público-Privada).

The 5 Computer Vision Techniques

clique na imagem para seguir o link

clique na imagem para seguir o link

Boa introdução ao tema da visão por computador

The 5 Computer Vision Techniques That Will Change How You See The World

Computer Vision is one of the hottest research fields within Deep Learning at the moment. It sits at the intersection of many academic subjects, such as Computer Science (Graphics, Algorithms, Theory, Systems, Architecture), Mathematics (Information Retrieval, Machine Learning), Engineering (Robotics, Speech, NLP, Image Processing), Physics (Optics), Biology (Neuroscience), and Psychology (Cognitive Science). As Computer Vision represents a relative understanding of visual environments and their contexts, many scientists believe the field paves the way towards Artificial General Intelligence due to its cross-domain mastery.

So what is Computer Vision?

Tags: ,

Basketball Stat Cherry Picking

clicar na imagem para seguir o link

clicar na imagem para seguir o link

Deep into the NBA playoffs, we are graced with stats-o-plenty before, during, and after every game. Some of the numbers are informative. Most of them are randomly used to illustrate a commentator’s point.

One of the most common stats is the conditional that says something like, “When player X scores at least Y points, the team wins 90 percent of their games.” It implies a cause-and-effect relationship.

The Cleveland Cavaliers won the most games when LeBron James scored 30 or more points. So James should just score that many points every time. Easy. I should be a coach.

It’s a bit of stat cherry picking, trying to find something in common among games won. So to make things easier, and for you to wow your friends during the games, I compiled winning percentages for several stats during the 2017-18 regular season. Select among the star players still in the playoffs.


50 Great Examples of Data Visualization

clique na imagem para seguir o linkBons exemplos de representações gráficas

Wrapping your brain around data online can be challenging, especially when dealing with huge volumes of information.

And trying to find related content can also be difficult, depending on what data you’re looking for.

But data visualizations can make all of that much easier, allowing you to see the concepts that you’re learning about in a more interesting, and often more useful manner.

Below are 50 of the best data visualizations and tools for creating your own visualizations out there, covering everything from Digg activity to network connectivity to what’s currently happening on Twitter.

Tags: ,