There are two different senses of complexity.

The one that is almost universally used is the complexity of parts. We say a computer is very complex, while a block of silicon is not. In this sense, complexity has to do mainly with quantity, i.e., parts one next to another in a whole. The dominance of this definition is not a random accident but arises because mathematics, which studies quantity, is at the heart of the modern scientific method. It is very natural for a culture built on modern science to think of everything in terms of quantity. Thus, it is very natural for us to think of complexity as an arrangement of extended parts as it is very natural for us to think in terms of extension, that is, in terms of physical parts one next to another.

Secondly, something is complex if it is hard to understand. It is simple if I can understand it with ease. More precisely, a thing is simpler when it is more intelligible, more understandable in itself, not necessarily just to us. But, we, of course, start with what is commensurate with our understanding to see what we mean by intelligibility, i.e. understandability. When is something simple to understand? Well, it depends on how much we know and how much time we have spent trying to understand it. What I’m talking about here is after we have put in all the time and thought required to understand something. Only after we have invested a sufficient amount of time and thought can we say whether something is really simple or not. To a child everything appears complex, even basic addition. However, the first time a child really understands 1+1=2, he sees it is very simple.

Extracted from Complexity versus Simplicity by Anthony Rizzi, Institute for Advanced Physics