Rubin will start a company to support startups interested in building technology-hardware products, Google said in an emailed response for comment on a Wall Street Journal report about his move.
James Kuffner, a research scientist at Google and a member of the robotics group, will replace Rubin, the company added.
Last year, Google's browser and applications chief Sundar Pichai replaced Rubin as head of the Android division, bringing the firm's mobile software, applications and Chrome browser under one roof.
Rubin built Android into a free, open-source software platform now used by most of the world's largest handset manufacturers, from Samsung Electronics Co Ltd to HTC Corp.
Sundar Pichai: The genius who could be Google's next CEO
Android One may not be a success in India
Is Pichai being groomed for top job at Google?
Google sees e-commerce swelling up to $40 billion by 2020
Xiaomi raises India sales target