Archive

Archive for December, 2012

Industrial revolutions

27.12.2012 Leave a comment

Paul Krugman discusses Bob Gordon’s essay in which Gordon describes three industrial revolutions humanity went through in the last few hundred years. Gordon suggests we’re at the end of the third industrial revolution and the fourth one is awaiting us in the coming decades. The suggested fourth industrial revolution will introduce robots in manufacturing and millions of people will lose their jobs. All the money from the lost jobs will go to the rich who will own the robots.

In my opinion bunching various inventions together into distinct industrial revolutions is too broad of a generalization. Certain inventions did influence quality of our lives, such as hygiene, tap water or vaccines. Other inventions changed the way we travel, such as steam engine or jet engine. But all of the minor “revolutions” overlapped and we can’t just draw a straight line between them.

The suggested future, the one which is coming very fast, in which robots will take over manufacturing, is not that dark at all. Major manufacturers in the US are already planning bringing manufacturing back to the States by creating factories full of robots, which are cheaper even than the proverbial Chinese worker. More and more people will be losing their jobs, but an economy in which everything is dirt cheap but nobody has money to buy it is not going to be sustainable. Eventually tax and social laws will change, even in the US – although it’s hard to imagine for most Americans, the US will have to be come more like Europe – and taxes on manufactured products will increase while income taxes will decrease. In fact people will be paid money just for living in a country or state and the money will come from taxes on manufacturing.

In my previous post I contemplated the future of humanity. There is still a lot of inventions on the way before we get there. The inventions which we can already predict include:

  • Robots – used more and more at factories, stores and eventually in homes.
  • 3D printers – everybody will be able to download plans from the Internet and make their own goods at home.
  • Nanotechnology – from health through manufacturing through new kinds of materials, etc. – it will improve every aspect of our lives.
  • Brain engineering – will help us transcend our biological bodies and become interstellar beings.

There is probably a lot more of exciting and significant inventions awaiting us, of which we can’t even think yet, just like the Internet surprised everybody and was not broadly predicted.

Categories: Universe

Multithreading in scripting languages

26.12.2012 6 comments

Scripting languages and multithreading don’t go together. Sure, many scripting languages implement something which looks and works like threads, but in the end these are usually fake threads, which work like a single CPU thread and run on a single CPU core. Fake threads are of course useful, but they cannot leverage the full power of a modern CPU. Or GPU.

Today CPUs have many cores and GPUs have hundreds of little “hardware threads”. Typically a GPU has multiple cores, each of which can execute multiple threads simultaneously.

All this computing power cannot be harnessed in general purpose scripting languages. This is because implementing proper multithreading support in scripting languages is very difficult. At some point Mozilla dropped multithreading support in their JS engine as they thought multithreading in JavaScript was not needed at all and the complicated multithreading implementation was a drag on the engine’s robustness.

What makes many scripting languages useful is dynamic typing. Variables don’t have predetermined types, the types of objects they reference can change in time. But from CPU’s point of view objects in scripting languages are complex, it is difficult to make them modifiable atomically with regards to other threads.

However, it would be great to use a general purpose scripting language and be able to leverage multiple CPU cores as well as GPUs. Scripting languages are easy to use – this is their purpose – so they are perfect for simple tasks or prototyping.

Options

There are three general approaches to creating a general-purpose scripting language with support for robust multithreading:

1. Making the interpreter/engine of a scripting language thread-safe.

This is certainly doable, but in effect it would make the interpreter very slow. Python has been criticized for years for being slow even without real multithreading support.

2. Designing a new scripting language specifically for multithreading.

This is an interesting option. Imperative and functional languages have a bigger potential for this approach. It is very difficult to design a new programming language which will be easy to use, though.

3. Modifying an existing language and applying some limitations for multithreaded operation.

This is something inbetween the other two options. This is the most promising option. The idea is to not sacrifice too much of a language’s flexibility but still provide robust multithreading support.

All three approaches have to take into account several common limitations.

Variables

The most common classes/kinds of variables on which functions operate are: locals, arguments, closures and globals.

Local variables which are not closures to inner functions are always thread-safe, because they are local to the executing function’s context.

This can be complicated by coroutines, functions which preserve their state across multiple calls. Local variables in a coroutine are no longer thread-safe if two threads want to call the coroutine simultaneously.

Function arguments are something between closures and locals from the programmer’s perspective. An object passed from the caller to the callee is accessible to both. If the callee is used as a thread function, the arguments are shared between two threads.

Closures share some similarities with function arguments, although they are in a way local variables to the function they belong to. Multiple threads may get spawned using a local function which has access to the closures. This way multiple threads may be trying to modify the closures.

Globals are the most susceptible to manipulation from multiple threads. Globals are not recommended in general, although they are typically more pervasive in scripting languages, esp. in languages where functions are first class objects and share the global namespace with other objects.

Immutability (or constness) is a very useful trait for variables. Immutable variables, or rather their immutable values, are thread-safe, because no thread can modify them, so all threads are safe to read them simultaneously. Therefore immutability is the ultimate key to thread safety. Unfortunately most scripting languages do not support explicit immutability. It is difficult to impose and control immutability in a dynamically typed language. Immutability would become another source of unexpected exceptions.

Possible solution

An example possible solution based on option 3 from the above list would be to leverage an existing language model but limit data exchange between threads. If threads cannot exchange data beyond special facilities like mailboxes, the risk of deadlocks or data races is low. It would work as follows.

All objects would have an additional binary state attribute, they would either be unique or shared.

A unique object is an object which is only accessible to the current thread and the thread can do anything with the object.

A shared object is an object which is accessible to multiple threads. A shared object is immutable and cannot be modified by any thread.

And here is how this new state would be used:

  1. When a program or script is started, all objects are marked as unique or shared or whichever is more convenient in the long run. In languages which have explicit immutability, immutable objects would be marked as shared. Otherwise all new objects are marked as unique.
  2. When a new thread is spawned, first all objects accessible to this thread are marked as shared. This is a deep operation and applies to all children of the global object and their children and so on. It also applies to the locals of the current function and all closures accessible to the current thread, as well as all saved coroutine state (continuations). Both the parent and the child thread proceed as usual once the child thread is spawned.
  3. Because shared objects are immutable, whenever a thread attempts to modify a shared object, a shallow copy of the object being modified is created, and that shallow copy is modified. The new copy is marked as unique.

Critique

There are two problems with the above approach. First of all spawning a new thread is quite expensive, because it requires walking all data visible to the parent thread and marking it as shared.

Secondly, the operation of shallow-copying shared objects to make them unique (mutable) is non-trivial, because it would require modifying all other objects which reference them as well in the same manner.

This second problem could be avoided if objects were always referenced by handles and the actual object pointers would be stored in a big array per-thread to which the handle would be an index. Only the object under the handle would be copied and made unique. Other threads wouldn’t notice this change.

The handle/array approach would also reduce the impact of the first problem, because instead of walking a tree of objects the interpreter would only have to walk a linear array.

The handle/array approach would have a negative impact on performance of modern engines where raw pointers are used internally to reference objects.

Alternative approach

Another approach would be to use thread identifier to indicate which thread owns a particular object. If the thread identifier stored in a particular object would be equal to current thread’s identifier, it would mean that the object is unique and mutable and can be modified by this thread. If the identifier was different from the current thread’s identifier, the object would be considered shared and therefore immutable.

When a thread spawned another thread, the parent thread’s identifier would be modified. This way all objects belonging to this thread would automatically be marked as shared. The objects would be directly inherited by the child thread without any complicated operations.

Now when either the parent or the child thread tried to access an object, it would still have to do a shallow copy. It is not worth mentioning that the new copy would be marked with the thread’s identifier, because all objects created by a particular thread would always be marked with thread identifier of the creator thread.

With this approach objects would still have to be referenced by handles. Therefore a copy of the handle space would have to be made for each thread when a thread is spawned.

Summary – impact on the interpreter/engine

  • All objects handles would have to be translated to access actual objects.
  • To mitigate the impact of object handles, local objects in functions could still be held by pointer instead of by handle.
    • This would be OK even if callees spawned threads which had access to the local variables, because spawning a thread would turn the objects into shared, so the local objects would have to be replaced if they were modified later by the parent thread.
    • However all handles would still be kept up to date for closures as well as in coroutines.
    • Global objects and properties of local objects would still have to be translated.
  • Object refinement would remain the same. Reading object properties would be unaffected.
  • Object modification would involve checking object owning thread identifier and creating a shallow copy of modified shared objects for the current thread.
  • Each thread would have its own handle space.
  • Creating a new thread would involve copying handle space from the parent thread.

Request for comments

Do you have any ideas how to further mitigate the penalties of using object handles?

Or can you think of a better general approach to the problem of multithreading in scripting languages?

Categories: Computing

Dwarf galaxies

15.12.2012 1 comment

I recently read that our humongous Galaxy was probably small in the beginning, but in time it accumulated stars by colliding with other galaxies. This conclusion was drawn from the fact that the outskirts of our Galaxy are composed of very old stars, which have the same composition as stars in dwarf galaxies. Our Galaxy is surrounded by lots of dwarf galaxies, we’ve just started discovering them. There’s at least 30 bright dwarf galaxies in our neighborhood which we already know and many dim ones are probably lurking in our vicinity. These dwarf galaxies contain very old stars, judging by their composition – these stars are very poor in chemical elements heavier than lithium, so they must originate from times when there was very little heavier elements in the Universe, or none at all. It turns out that the stars in the outskirts of our galaxy carry similar spectral signature.

From this I drew two conclusions.

Where to search for life

First of all there is very little sense in looking for life in dwarf galaxies or in the outskirts of our Galaxy, or any other behemoth. The areas which are poor in heavier elements are likely to have low probability in harboring life, because of the deficiency of the needed chemical elements. So the best place to start the search for other life forms is in the richer areas of big galaxies such as ours, or the Andromeda Galaxy which we are going to collide with in a few billion years.

This reasoning may be flawed. I am yet undereducated in the topic of how stars are forming. My guess is that the light gas contracts easier than heavier elements, which instead form clumps around the forming star and turn into planets after a few million years after they sweep all the dust in their orbits.

On the other hand I suspect that the more adventurous places of space are better for evolution. Our spiral Galaxy is thought to be moving its “top” side forward and our star is oscillating between the “bottom” and the “top” in cycles of around 70-90 million years. Whenever our solar system is near the “top”, it is exposed to the influence of the intergalactic medium. This probably induces recurring cataclysmic events, such as the big extinction event 65 million years ago. After every such event the conditions on our planet change significantly and wake up the evolution. Let’s put it this way: the dinosaurs dominated on Earth for about 135 million years and didn’t accomplish much. We, mammals, accomplished much more in only 65 million years after we’ve been given a chance after some rock leveled the evolution’s playing field. Similar cataclysmic events may have triggered the plants and animals moving out of the water to conquer the land and who knows what else.

If you look at the dwarf galaxies, they are dull. They have survived probably 10 or more billion years and remember the times when the Universe was poor in elements needed for life. If they were off to an adventure, they would have already merged with some big galaxy or became one on their own. No, they are small and full of old stars.

I am not claiming that there can’t be any life in these areas where old stars live, but the odds for life are much better in the rich parts of big galaxies.

Locality

The other conclusion I drew from the article is that all we have in our Galaxy and the surrounding galaxies has always been here. In the beginning when our area of the Universe formed it was filled with gas. The gas contracted into lots of small galaxies. These galaxies started merging. Two of them started consuming others and became behemoths – our Galaxy and the Andromeda Galaxy. The rest remained small and is probably slowly rotating around the center of our local cluster, which lies in the middle between the two behemoths. I suspect that all the material in our local cluster is the same as it was in the beginning. There was no or little exchange of material with other clusters. We could consider Andromeda as well as all our local dwarfs our siblings.

Not that it matters, the composition of stars in other clusters probably follows very similar patterns. We will eventually start exploring these too and know more about them. But I suspect the only way for galaxy clusters to exchange material is through civilizations capable of intergalactic travel.

Categories: Universe

Alternative point of view on C++11

13.12.2012 Leave a comment

Everybody loves C++11. What’s there not to like? In fact everybody I know wants to switch to C++11 at the first opportunity, including me. This is because C++11 is a better language than C++98. Or is it?

John Sonmez argues in his post that besides all the great new features which make C++11 feel like a new language, there is one big problem with it – it’s HUGE!

I agree with John’s statement, C++ has grown big. It’s been hard to learn and master. It’s even more complex now. I’ve been learning C++ for many, many years, and it still surprises me sometimes, and I haven’t started using C++11 in everyday code yet.

There are lots of very nice features in C++, but there are also lots of features which are tricky. There are features which invite bugs. Sure, friends, multiple inheritance and even goto all have their place, but in most cases they will make someone miserable in the long run. And it’s not about “don’t like them, so don’t use them”. It’s about all those unexperienced programmers who stumble upon them and think they are a good idea to use.

One possibility to improve this state could be to remove, limit or forbid certain features. Subsetting is discussed every now and then. As good a solution as it may sound, it won’t solve all the problems. Some suggest the problem lies in C++’s compatibility with C. So subsetting definitely won’t solve this, as the remaining features will need to remain unchanged.

I suspect Andrei Alexandrescu would say something like: “Don’t like C++’s complexity and C compatibility? Switch to D”. Yes, there’s that.

C++ just can’t break backwards compatibility. This is why new languages are created. One day some programmer or group will gather all the best ideas from C++ and create a new language, which will be simpler and will avoid many pitfalls of C++.

Until that happens, I will happily look forward to using C++11 in my projects.

Categories: Computing

Global warming

10.12.2012 Leave a comment

We may be in for a ride, bigger and sooner than we thought.

2012 was not the year the world ended as predicted by those who did not know how to interpret the Mayan calendar, but there were a couple of disturbing climate changes. The most recent data suggests that there is a lot of carbon locked in the permafrost – more than scientists estimated so far. Let’s add to this that this year more ice melted in the far north and in the Arctic ocean than ever before (well, at least in the last few hundred years).

Maybe next year, maybe in a few years during summer on the northern hemisphere more ice will melt and the carbon will be released in the form of carbon dioxide and further accelerate global warming.

The recent climate hiccups are the result of the climate being knocked out off an equilibrium. Even before, the climate was getting a little bit warmer year to year, but it was slow and gradual. Many people remember snow on Christmas in the 1980s. Then came the 1990s and there was barely a few days of snow in the winter. In the recent years winters are either mild or strong. Summers are either very hot or mild or rainy. Similar thing is happening to El Nino – it’s accelerating and gaining on strength. This is because the global warming is accelerating and the global climate is unable to keep up resulting in hiccups.

Given what happened this year and what we’ve learned, the global warming is probably going to kick up a notch, more than we anticipated. In the next 10-20 years we will see the global temperature increase a few degrees centigrade and the sea level rise.

Rising sea level is of course bad for many people, because a large amount of world’s population is living close to the sea shore. My advice is, if you are planning to buy a house by the sea, don’t. If you live by the sea, sell the house and move inland, otherwise you will lose it. The rising sea level will cause massive losses and will impact global economy. It may result in a global crisis worse than the one which started in 2008.

It’s difficult to tell when it’s a good time to start preparing, but weather and climate are ones of those things which are super difficult to predict.

Categories: Universe

The Universe is taming with life

8.12.2012 Leave a comment

For decades we’ve been wondering if there are other planets out there, let alone other life.

Finally there is some positive research which concludes that our solar system is not exceptional. In fact, other nearby systems may have better conditions for life than ours!

There are 8 planets in our system, 3 moons which can rival Mercury in size, plus several other sizable moons. You could say that our planetary system is rich in celestial bodies. At least three planets in our solar system could harbor life – Venus (if not the acid), Earth (well, there is life on it) and Mars (eventually we’re going to find out if there’s life on it or was in the past). Some moons of Jupiter and Saturn could also potentially host primitive microbial life. So many opportunities in one planetary system!

There’s nothing special about our star. Probably many if not most stars have rich planetary systems. For more than a decade we’ve been detecting new planets around other stars, there are already hundreds of planets known and most of them are even bigger than Jupiter. Big planets tend to have many moons – more opportunities for life even if the big planets are far from the star.

The article suggests that the habitable zone around many stars may be even bigger than around our star.

The conclusion – the Universe must be taming with life. Of course intelligent life is another story as it needs hundreds of millions of years to evolve from primordial soup.

Categories: Universe

The future of the human race

7.12.2012 2 comments

In his recent post, Scott Adams envisions that just like in popular literature and movies, we will create a race of robots, which will eventually seek to destroy us. Then another race of half-robots-half-humans will come to existence and save humanity.

I do not fully agree with his prediction, but there is a little merit to it. Let me explain what my stance is.

You may disagree with what I am going to write and that’s understandable. The idea of annihilation by AI or robots is well-known and scary to most people. Yet I do not think that we are in danger, but rather that great opportunities lay ahead of us.

Human brain is a wonderful and quite complicated computing device. In the years to come we will get to know it much better and eventually we will figure out how it works, even though today our knowledge of its workings is very limited. Recently I’ve read an article about new discoveries how the neural networks in our brain work. What makes it so fault tolerant is that when single neurons fire, they are ignored. Moreover many neurons can randomly fire and nothing happens. But when information is incoming on the input receptors, such as eyes, ears or skin, all relevant neurons fire simultaneously and carry the information through the network like a wave. This is when the information processing and thought processes occur.

With further advances in nanotechnology and better understanding of the brain we will eventually be able to create brain prosthetics, implants and eventually convert a brain into a non-biological device. The procedure will probably consist of injecting nanobots into the blood stream. The tiny devices will reach the brain and will start replacing neurons with artificial devices. It is also possible that an implant could be installed under the skull and perform the procedure, but injection of nanobots will certainly be less invasive and injecting nanobots into the bloodstream to repair the body, perform medical procedures and diagnostics will likely be common by that time.

Eventually the brain will be fully converted into a non-biological device. It will still be connected to the rest of the body, all the inputs and outputs. It may still be taking energy from the bloodstream or heat, or a new infrastructure will be installed to power it. Possibilities for powering the artificial brain will be many.

The artificial brain will be much better in many aspects than a biological one. It will need less energy, so it will be more efficient, it will be much faster and it will not be susceptible to natural death. A person with an artificial brain will initially be the same as before, will think the same, will have the same personality, the same memories and feelings. It will be exactly the same person.

After a short period of time a person with an artificial brain will see significant improvements in the perception of the world, thinking processes, etc. The brain will adapt to its new capabilities. This process will have to be guided by a psychologist, because the person will become super smart over a few days, so the process of becoming a new being will be difficult and may lead to a collapse if not carefully controlled.

Having an artificial brain will open endless possibilities. A new engineering field will open up – brain programming. New programs will be created for artificial brains, programs which will enhance the owners of the brains immensely with mental abilities we can’t yet imagine. Of course the term programs may not be appropriate, since a brain is a mix of hardware and software.

At this point computers as we know will seize to exist. Nobody will use laptops and smartphones. There will be no need. Such primitive computing functionality will be easy to integrate directly with the brain and it may even happen before artificial brains are invented in the form of implants. This will also open a serious problem of brain hacking. To mitigate it, connectivity will be limited only to certain parts of the brain and special programs will run continuously to detect and shut off or reprogram compromised parts.

Learning will not be an issue anymore and will be effortless. It will be easy to store Wikipedia and beyond in the brain by just downloading the content directly to memory.

The size of the brain will be substantially reduced keeping the same functionality. Normally we don’t use all parts the brain, they are idle. Therefore it makes a lot of sense to compress and shut down the unused parts or programs of the brain. This will be easy with artificial brains. A brain of the size of a mouse’s will be no worse than a brain of an adult human. The added benefit of a small brain is that it requires less energy.

Mental abilities aside, it will be possible to swap bodies. Because the brain will now be artificial and will not rely on body chemistry, it will be simple to disconnect it and connect it to and artificial body instead. Some people may choose to freeze their original bodies, or simply leave certain portions of the brain which control the autonomous systems in the body. They will wear their artificial body to work and in the evening swap for the biological body to enjoy life.

Connecting brain with an artificial body will initially be difficult, due to the fact how the brain is wired. Our brain is used to the nerve system in our bodies, disconnecting it and connecting different kinds of inputs and outputs may result in a shock. Eventually programs will be created which will make this easy and it will just feel normal.

Having both and artificial brain and body will open new possibilities to us. Any kind of work will be possible. We will be able to live and thrive in any place of the world. We will be able to live in the depths of the oceans as well as the deserts and poles. Poverty and famine will disappear.

Artificial brains and bodies can be charged directly from the Sun. Notice that today we also take most of our energy from the Sun: we eat plants which take energy from the Sun, we eat animals who also eventually take their energy from the Sun just like us. We power our devices from the Sun as well – energy stored in coal and oil also originates from the Sun. According to Wikipedia typical efficiency of photosynthesis 3 to 6%. We will make it 30-50%. The skin on the artificial bodies will be able to extract energy directly from light. It will be sufficient to just stand in front of the Sun with open arms for 15 minutes a day – plenty of time to meditate or rest.

The sleep is an important piece of data acquisition. During sleep our brains sort the information and help us memorize what we learned during the day. Artificial brains will sleep as well, but they may as well sleep when we are performing other tasks. For example it will be possible to go to work and perform manual, repetitive labor while the part of the brain responsible for memory will be sleeping. Or we will just sleep when charging.

Ultimately we will be able to travel in space with unlimited acceleration and for unlimited periods of time. Artificial brains can be shut down for a very long time to avoid dying from boredom. Just like hibernation, but very easy to achieve. Artificial bodies will be crucial in going beyond the solar system. Biological bodies require very expensive and complicated life support systems, which are heavy and have a lot of mass, so require a lot of energy to accelerate them. The total mass needed to travel in space for biological beings is huge. Additional mass is needed for protection from radiation. Even more mass is needed to keep the bodies in shape, even if hibernation technology is invented. Even if all that extra mass was not necessary bodies are not able to withstand acceleration exceeding a few g. Therefore traveling in space is economically infeasible for biological species.

But artificial brains and bodies do not have these limitations. In fact a spaceship will be small, it may as well fit in a pocket. Thanks to brain compression technology a traveller will insert his compressed brain into the starship along with nanobots needed to rebuild the body at the destination. A small spaceship will not require as much energy to move and can accelerate very quickly. Traveling to distant stars an even galaxies will not be an issue. It will be as easy as downloading the plans of the spaceship from the Internet, building it with your own nanobots which are in the body and bon voyage!

The creatures with artificial brains and bodies will still be us, it will be our grandsons and granddaughters. However they will no longer be humans. They will be a new species, born on Earth, just like we are the children of this planet. Their goal will not be to annihilate us, just like it’s not our goal to destroy the life on Earth. They will look into the future and into the stars, they will be discoverers, thinkers and pilgrims. They will carry their legacy to other civilizations and join them in exploring the Universe.

Given the rapid evolution of our technology over the past few hundred years it is likely that this will happen soon. The pace of innovation is accelerating, more countries like India and China are helping us push science forward.

Likely it takes about a few thousand years for a civilization since the invention of basic technology and science to advance to the point at which they can turn themselves into a new species, leave their planet and explore the Universe. It’s a big jump and in the history of a planet it does not take long, but it takes a long time for evolution to get to this point.

Probably the Universe abounds with advanced civilizations. Sometimes species evolves far enough to create a civilization. Once this happens, it takes a few thousand years to become the citizens of the Universe. There are no limits how artificial bodies look and work, so they are very diverse, but in principle all intelligent species look the same regardless on which world they evolved. I suspect they mix their technologies and eventually you cannot tell where they originate from, it does not matter anymore which planet their ancestors came from. Just like today humans mix and especially in Western countries people have ancestors from all over the world.

We’re constantly asking ourselves if there are other civilizations. We’re trying to search for them. But it makes no sense, the probability of finding a civilization on the same level of advancement as ours is very, very low, because all civilizations quickly evolve into interstellar, non-biological beings. Likely these civilizations do not want to contact us just to see how we evolve and they give us the opportunity to get there on our own. These explorers may be sitting around for millennia undetected enjoying the history of our planet. Or maybe they are just coming by from time to time just for fun, visiting various distant worlds.

The idea of dropping biological brains and bodies may sound disgusting or mind boggling to you today. But the change will be gradual and will take many years. People will start switching to become better, smarter, less reliant on biological bodies, or perhaps just because its cool and interesting. Eventually there will be fewer and fewer people with biological brains. The sociology will come into play and those who don’t switch will be treated as stupid and old-fashioned. It will be OK to have a biological brain. It is unlikely that there will be wars because of this, wars do not make any sense and people with artificial brains will be intelligent enough to think of better things. People which will choose to keep their biological brains will use the technology to extend their bodies anyway, just like about everybody today has a cell phone. Children of people with biological brains will be more eager to switch though. Eventually all people with biological brains will die out. It may take a few hundred of years, but the new species will eventually supersede and replace humans.

Do not be afraid. The future awaits.

Categories: Universe