Takuya Kitazawa

Act on Complicated Things

Mon, 01 Jun 2026 00:00:00 +0000

Our world is beautifully complex, but it doesn't necessarily have to be as complicated as it is today.

Complex vs. Complicated

Being complex and complicated are fundamentally different. The former is always unpredictable, dynamic, interdependent, and self-organizing by nature. Numerous factors influence one another, and the output for a given input is probabilistic, if not completely random. There is no fixed rule of saying, "If you do this, that always happens."

Climate change, global politics, the economy, and the development of social creatures like human beings are all complex. Nothing guarantees any single consequence, and the phenomena can only be understood by the most reasonable likelihoods derived from what we already know. Yet, it is practically impossible to know everything due to the astronomical invisible factors. (Supercomputers and quantum algorithms might increase the odds by handling more data efficiently, though.)

The complicated things, on the other hand, show linearly predictable behaviours. They are like entangled strings; even if it seems impossible to decompose, the object is eventually a sum of simpler, static building blocks. As long as the input remains the same, a complicated system will consistently generate a similar output with minimal uncertainty. No matter how lengthy and messy they are, the intermediate logics can be written as a set of rules, i.e., algorithms, like factory machines and computer programs.

What I'm Doing Now

Sun, 31 May 2026 00:00:00 +0000

This is a now page. I will update the page roughly once a month to share where I am, what I'm focusing on, and what I don't do.

It's a beautiful season in Ottawa! The city is so green, flowers are blooming, and the temperature is moderate.

Search for a new (remote) opportunity

It's been 8 months without big trips (very unusual for me, except the COVID period), and the eventless life is likely to continue for a while for personal reasons. Even though I like maintaining daily routines in quietness, the condition also brings me a subtle sense of boredom and dissatisfaction. My freelancing work is also more stable nowadays, which is good and bad. So, I'm now open to work. I genuinely miss the exposure to complex problems and the serendipitous interactions with others in customer-facing work.

5 months of practicing French

I've consistently spent 3+ hours per day in learning French since January, and I think it has been improving at a decent pace. In particular, reading online articles, writing blog posts, and listening to not-too-fast podcasts (like this and this) have become more effortless. Speaking, on the other hand, is still far from conversational fluency. Even though I'm working with an amazing tutor on italki multiple times a week, I'm feeling the plateau, and I definitely need more opportunities to talk with people in the real world.

A Recipe for Sustainable Data Products

Fri, 01 May 2026 00:00:00 +0000

By and large, digital systems consist of two different parts: data and information.

This website, takuti.me, for example, is composed of a combination of numbers, text, images, and links. They exist in the digital world as data, which can be represented by the binary digits 0 and 1. Think of them as raw ingredients of a meal; each of them can be something yet unsophisticated to us as-is.

On the other hand, information is the result of processing and organizing data in a meaningful way. It is a menu, dish, and course that you will eventually come to appreciate and savour. I need to intentionally put the ingredients into context (i.e., recipe) so that deliverables are useful and relevant to a specific purpose or goal. Hopefully, takuti.me effectively visualizes my identity and conveys certain messages to its audiences as such.

Yes, there are a lot more things that happen in the kitchen, such as version control, hosting, networking, research, optimization, personalization, and interface design. But the tools, methods, and processes are to be chosen in a way that makes sense to the given data and purpose. (If you disagree with this point, you may want to examine the idea of ethical product development.)

Toward Inclusive Education

Wed, 01 Apr 2026 00:00:00 +0000

There are many educational practices out there.

Recognizing the complexity, communities proposed a methodology called Design-Based Research (DBR). In contrast to traditional studies conducted in controlled laboratory settings, DBR contextualizes academic research by placing researchers alongside students and teachers as collaborators. Through an iterative design and feedback process in the field, DBR aims to yield authentic and useful knowledge in classroom practices.

As such, education scholars are working hard to streamline best practices, make tactics actionable, and keep things up-to-date. And, especially in the Global North, teachers are very much aware of what is desired in theory.

To the best of my knowledge, the following components signify modern pedagogy.

Digital tools, as society actively invests in infrastructure development and skill training projects. It is now necessary to respond to the technological advancement in classrooms.
Cultural understanding to overcome realistic constraints, acknowledge local contexts, and develop long-lasting, tool-agnostic skills in the information age. In fact, digitization may be optional in certain situations to achieve an educational goal.
Agency for what Brazilian educator Paulo Freire called critical consciousness and problem-posing education, which center learners and encourage them to participate in educational projects actively. It is also the key to bridge the missing link in the digital divide.

The incompleteness of computer education in low-income countries can be explained by the lack of cultural understanding and learners' agency, for example.

Who is Digitally Prepared?

Sun, 01 Mar 2026 00:00:00 +0000

Technology is contextual, and hence neither oversimplification nor overcomplication is helpful; it'd be advisable to understand realistic constraints in a very environment and take a tailored approach point by point.

But how can we "see" real challenges and opportunities a person, organization, community, country, or region faces in the information age? Where are you currently at, in terms of digital divide?

One way to evaluate this is to travel to a field, spend time with locals, and co-explore problems and solutions. Such fieldwork, however, can be resource-intensive, uncertain, and irrelevant to other contexts. Consequently, the work is likely to be unsustainable with minimal scaling-up potential.

That's where assessment frameworks come in.

Task-based assessment

By applying a standard set of tasks and measurements globally, institutions can gain qualitative and quantitative insights about technology adaptation while eliminating noise. This eases a local reality check and cross-contextual analysis.

For example, International Computer and Information Literacy Study (ICILS) is a worldwide assessment of ICT literacy in educational settings conducted every five years. They employ their standardized questionnaires and virtual hands-on tasks to measure participants' ability to utilize ICT.

Professional Services

Thu, 19 Feb 2026 00:00:00 +0000

As a technical product builder based in Ottawa, Canada, I have provided the following services to individuals and organizations worldwide since 2022. See my portfolio to learn more about my work, and feel free to contact me for a casual inquiry. Terms differ depending on a client's needs, location, and currency.

What to expect: My work is informed by the principles of the ITIL framework and human-centered technology practice, which emphasize solutions grounded in each client's context, priorities, and operational realities. Hence, I take a collaborative, outcome-focused approach, working closely with you to iteratively shape deliverables through ongoing feedback and shared decision-making. This ensures solutions are practical, responsible, and sustainable, and that they align with how you actually operate over time.

Product Development

Guide architectural and strategic decisions on software development projects
Domain-specific advisory and audits in recommender systems, AI, data science and analytics, and machine learning productization
Knowledge transfer and training in data-driven operations and product management practices

Digital Transformation

Lead organization's digital transformation, including strategic planning, IT infrastructure design, digital marketing practices, and people development
Onboarding and evaluation of data-informed digital systems and operational procedures
Targeted capacity-building activities through presentation, hands-on training, and documentation

Digital Inclusion and Participation

Assessment of technology development (ICT4D) initiatives run by non-profits or public-sector teams
Technical advisory for digital inclusion programs and their M&E activities
Strategic partnership to cultivate digital infrastructure, literacy, skills, and problem-solving and critical-thinking practices in marginalized communities, especially in low-resource or low-connectivity contexts

Mentorship

Both short- and long-term engagements with mentees having a clear objective in tech or career.

The AI Divide

Sun, 01 Feb 2026 00:00:00 +0000

People worldwide exhibit varying levels of excitement or concern about AI, from the rich to the poor, ordinary citizens to global leaders to politicians, regardless of their occupation and industry. One cannot avoid the wave, even if "I'm not an expert" or "I don't use computers."

Therefore, before further investing in the technology, it is essential for each one of us to face reality, understand both the benefits and risks, and make informed and intentional decisions.

To be more precise, I often start my conversation with clients and audiences by highlighting the following points:

AI, like any other technology, is context-dependent.
AI represents the existing power dynamics and inequality.
AI can deepen the digital divide, if not accompanied by users' ability to think critically and solve complex problems.

First, the impact of technology use depends heavily on the context.

AI in Egypt and Uganda, for example, requires different approaches depending on the countries' demographic, geographical, political, cultural, societal, and economic conditions. Even within the same country, rural-urban inequalities block one-size-fits-all technological interventions. In the end, you and I always want separate prescriptions for technological advancement.

Why Offline Learning Still Matters in 2026

Thu, 01 Jan 2026 00:00:00 +0000

Short answer: There are two extremes in our world—hyper-connected world of rich countries, and a disconnected world of underserved communities like rural Africa.

I've seen both, and I conclude that being in a low-resource and low-connectivity environment was neither a disadvantage nor a limitation if they could receive localized, tailored services.

Being online is a privilege

First of all, not everyone enjoys the online environment. More than 25% of the global population still doesn't use the internet¹, and the disconnected population can reach around 80% in low-income countries, like Malawi².

That's why we need a different approach to achieve a similar goal. Education, for example, cannot rely on tablets, Zoom, or Google Docs in many parts of the world, yet we can still deliver a remote learning experience through low-tech applications like USSD.

Learning initiatives fail when we apply the rules of the hyper-connected world to the disconnected world without adapting to its context.

Hands-on needs are in the field

Moreover, context adaptation is only possible based on an accurate, vivid understanding of the context, which requires extensive fieldwork and interviews. It is an in-person process of learning who, where, which, why, what, and how a certain problem emerges and evolves.

Missing Link to Bridge the Digital Divide

Fri, 05 Dec 2025 00:00:00 +0000

Digital literacy and skills are essential anywhere across the globe, but especially in developing countries where socioeconomic opportunities are limited. In the constrained environment, ICT can be a game-changing tool that eliminates physical boundaries and unlocks numerous possibilities in and beyond communities of the oppressed.

Thus, governments and organizations actively invest in technology development in the following forms.

Infrastructure: Internet penetration remains a key issue among us. We have policies and strategies, and continue to invest in the basic needs in collaboration with the private sector, like telecom companies.
Funding: Don't have money and equipment in the households and communities? No worries, we support the financing and procurement of phones and computers. We have funds available.
Training: Let's teach digital skills, such as typing, coding, and using AIs, to marginalized people. We run many programs across the countries.
The Impact: Employability of the target population increases, and they will eventually land jobs and become self-resilient. That's the way to an equitable society and poverty eradication!

This is, in fact, the narrative behind a lot of development initiatives and success stories focusing on digital skills. For example, the World Bank reported the positive impact of "digital skills for jobs" projects in the Middle East and North Africa context. The initiative doesn't seem different from the problematic approach I witnessed in southern Africa, though.

"We," Not "I"

Fri, 07 Nov 2025 00:00:00 +0000

The more senior I am, the bigger my work is. And the larger the problem is, the more I feel powerless.

I cannot change the complex systems. But, we can. That's one of my greatest realizations while pursuing my career in tech and philanthropy.

Power of Collective Actions

It's all about scalability.

Yes, I can work directly with some individuals on a certain topic, like I do for mentorship. The subjects (e.g., mentees) will certainly grow, but it doesn't change the big picture of the world, such as today's market landscape and technological trends. Eventually, I can only see a handful of people among millions who share the same problem.

I can also give my finite resources, such as money and time, to a cause I care about, and it will be better than nothing. I may even become a life-changing partner for those who used to be in a challenging environment, and my knowledge and skills possibly influence the trajectory of a meaningful project. However, climate change, poverty, and inequalities won't stop anyway.

Uche Safe App

Wed, 01 Jan 2025 00:00:00 +0000

Scope

Served as the sole product partner to Uchembere Wabwino Maternity Clinic for designing and implementing a digital health learning application. Defined UX/UI and technical requirements for low-literacy and offline environments in Sub-Saharan Africa, enabling onboarding of 30+ non-technical users for concept validation.

Key Features

Optimized for Low-Literacy Users: Provide a step-by-step, bite-sized series of lessons for Sexual Reproductive Health and Rights (SRHR), accompanied by interactive quizzes and engaging illustrations
Offline-First Design: Requires no internet access after installation, with optional behavioural tracking through Google Analytics and BigQuery for monitoring, evaluation, and personalization
AI-Enabled Prototyping: Built on rapid iterative product cycles using AI-assisted prototyping tools such as Canva AI and Claude Code, ensuring timely delivery and feedback triage to meet the time-sensitive and mission-critical nature of the app

Testimonial

Working with Takuya was an absolute privilege. He is a rare tech professional who combines deep technical expertise with reliability and trust. He transformed our vision into a fully functional, user-centred product and successfully led the launch of our app on the Google Play Store. An over-the-moon tech partner you can count on to deliver excellence.

Contact Me

Mon, 21 Oct 2024 00:00:00 +0000

For an inquiry, I can be reached at hello@takuti.me.

Alternatively, you can go ahead scheduling time with me at Calendly:

The End of the Beginning—What I Talk About When I Talk About Malawi

Mon, 24 Jun 2024 00:00:00 +0000

"How's Africa?"—my friends outside of the continent frequently ask.

Where should I start...?

Malawi—the 10th poorest country in the world¹, with a life expectancy of around 60 years², less than 30% internet penetration, a median age under 18, and more than 80% of the population living in rural areas³. It's a country far from discussions about race and ethnic diversity and predominantly uses underrepresented languages. This is the "Africa" I have lived in for the past 11 months as a Canadian international volunteer, and now I'm about to end my one-year assignment. Unsurprisingly, there are many immediate struggles you wouldn't experience in wealthier nations. But life here is actually not *that* bad and can even be happier than in more developed countries.

Let me be honest. I am exhausted, both mentally and physically. I don't know if it's due to the side effects of malaria prevention medicine, the full-time onsite work in a dark and noisy office space, the general safety concerns that hinder me from going out as frequently as I used to, or the people who stare at me whenever I'm outside. In any case, I've been struggling with fatigue and a sense of insecurity throughout the year. My body feels shockingly heavy (without any visible weight gain), and I cannot focus on anything, even when I need to. In a short period, I've completely lost stability in many aspects of my life, including mindfulness, fitness, finances, and intelligence. Despite this, the handful of valuable things I've received—such as connection, laughter, and joy—make me feel like these costs are worth paying. I have no issue with living in this country longer if necessary.

Materializing Digital Transformation

Mon, 27 May 2024 00:00:00 +0000

Start your digital transformation from the field, by understanding people. Please—your people are not here, in this big conference hall in the big city.

That's how I concluded my guest presentation at a conference organized by the ICT Association of Malawi (ICTAM) in the capital city earlier this month, reflecting on my engagements with Malawi's youth- and rural-centered population¹.

ICT is today's infrastructure—like roads, water, electricity—and literacy—like reading and writing. That is the power of the people, by the people, and for the people. Thus, the availability of technology is something very tangible, material, and life-changing, both positively and negatively².

You say "AI," for example. But AI for what? To whom? Where? When? How? These fundamental questions are often overlooked behind technological trends and advances. As a consequence, your "innovation" simply fails, if not significantly damaging surroundings. Downstream impacts of such failures can take multiple forms, such as consumer mental health challenges, exploitation and inequality, climate change, and loss of humanity at large.

I'm Not "China"—Connecting HERE and THERE in Systems

Tue, 30 Apr 2024 00:00:00 +0000

"China!" A random stranger yells at me. Welcome to Malawi—the Warm Heart of Africa—though I don't consider it "friendly" and it is simply racial discrimination¹. Over the last nine months living in the country, I've also received countless judgemental comments on my character, socio-economic status, and thought and behavioral patterns based on the stereotypes they might have against Japanese, Asians, and foreigners at large. Unfortunately, this kind of discomfort is the only yet non-negotiable reason why I cannot imagine my future living in this beautiful country; from my recent experiences in Ethiopia, Egypt, and Malawi, I learned being an Asian in Africa naturally puts me into an endless psychological tension with locals and brings a greater sense of insecurity and vulnerability. The alienated feeling strongly demotivates me to invest my short life in the environment.

* Malawi is undoubtedly beautiful. My concern, and future opportunities, inversely, are centered around its people and businesses.

Meanwhile, I was recently traveling to Frankfurt in Germany, Tallinn in Estonia, Helsinki in Finland, Stockholm in Sweden, and Copenhagen in Denmark for two weeks by plane, ferry, and train. Although I'm willing to write a full article about the trip, the biggest surprise was that nobody stares at me in the public spaces and shops and restaurants. In Malawi, on the contrary, I always feel someone is looking at me whenever I'm outside. Thus, it's been a while since the last time I felt it was okay to be my authentic self from the deepest part of my heart, and I simply enjoyed the emotional freedom during the trip. What a great pleasure running in the city center of a large Scandinavian city for 10km with an empty mind! As running is an essential part of my life, I could easily live here permanently.

Language: Behind the Power Dynamics in Information Society

Fri, 22 Mar 2024 00:00:00 +0000

Muli bwanji? ("How are you?" in a polite way, in Malawi's local language called Chichewa)

A police, immigration, or military personnel asks.

If you respond Ndili bwino. Kaya inu? ("I'm fine. And you?"), they smile. If you reply in English with confusion, like Ah...hello? there is a higher chance that the personnel will ask you to show your passport and explain what you are doing here.

Language is more than an interface to exchange semantics. It tells a lot more about the contexts, such as your identity, mood, and status. I recently learned that the Malawian officers try to efficiently read this information by greeting in the local language, even if they can speak English fluently. Thus, if we could understand and speak a local language, we could be united stronger. Otherwise, there will be more tension and hassle.

Languages represent the power

This experience reminds me of the history of the world's most widely spoken constructed language, Esperanto; it is recorded that its creator L. L. Zamenhof worked on the language to foster harmony between people from different countries. Eventually, language is the foundation of humanity.

One of the Poorest Life Is Not That Bad

Mon, 26 Feb 2024 00:00:00 +0000

Over the last few articles, I kept introducing Malawi as one of the poorest countries in the world. The statement is based on the World Population Review's "Poorest Countries in the World" report, and Malawi is ranked the 10th poorest country in terms of purchasing power parity (PPP), according to the World Bank-sourced data. But, what does the 10th poorest life look like?

Imbalance between earning and spending

First and foremost, as of February 2024, Malawi's monthly minimum wage for general workers is 90,000 Malawian Kwacha (MWK) both in rural and urban areas (!). That is, since 1,000 MWK is about 0.59 USD as of writing, we can expect to earn about 53 USD per month in Malawi at minimum. Yes, you read it correctly. It should be noted that a different lower rate may apply for domestic workers, and the World Bank once reported¹ that around 80% of the population is relying on agriculture for their livelihoods, occupying more than 60% of total employment. Hence, I would imagine the general population's income stream is fairly inconsistent in practice.

Relativize Malawi, and Rethink Their Contexts

Wed, 31 Jan 2024 00:00:00 +0000

During the holiday season, I traveled to Addis Ababa in Ethiopia, Dubai in UAE, and Cairo, Luxor, and Aswan in Egypt. It was an awakening experience visiting these big cities after staying almost half a year in Malawi, one of the poorest countries in the world¹. At first, I was excited to relativize the situations I observed in Malawi compared to more developed environments in North Africa and the Middle East. However, when our plane was about to land at the Addis Ababa Bole International Airport, I soon realized the effort in making a comparison between Malawi and the rest was essentially meaningless; the numerous lights shining under the sky in the city were completely at the different level from the Malawian standard, and my intuition told me it is impossible to directly project the brightness onto Malawi's future.

* The outside view near the destination was too different from the origin, and I impulsively took this photo from an airplane window.

My 2023 Annual Review: Between Individual and Systems

Mon, 25 Dec 2023 00:00:00 +0000

In short, the year 2023 has brought deepness to my life. It is exactly what I stated in the 2022 reflection as something I wanted to try next year, and I have committed to exploring the problems I have been concerned about for the last decade—skewed power dynamics and loss of humanity in the use of information technology. From January to December, by surveying academic papers, traveling to various destinations, interacting with mixed people, making my hands dirty in the field, and writing less frequent but more focused 12 monthly blog posts (vs. weekly 52 posts in 2022), I have continuously crystallized and reshaped my understanding of systemic issues. Now, I'm better equipped with the new tools to challenge the complex problems our society holds, both intellectually and practically.

* One of the year's last activities was to congratulate the Malawian young people who completed a computer programming course I have taken a trainer role for the past three months. Such experiences in the field are great opportunities for me to think deeply about and act against social issues I have been concerned about.

Is Computer Education Always Good?

Thu, 30 Nov 2023 00:00:00 +0000

Children are the hope, and they represent the future. That's what I immediately understood after becoming an international volunteer in Malawi, a small African country with more than 50% of the less-than-18 population. Thus, although it won't be the same, I was able to deeply relate my experience to Little Princes: One Man's Promise to Bring Home the Lost Children of Nepal, which illustrates Conor Grennan's volunteering journey started at an orphanage in Nepal.

To make the children's future brighter, one of the "low-hanging fruits" we can pick is an investment in education; ever since I started researching international development, I've seen a lot of young people-focused education and training programs funded by organizations and governments to promote equitable access to intellectual resources and directly empower the future generation. As a volunteer in the ICT domain in Malawi, I have also been part of these initiatives over the last few months with a particular focus on computer skills.

* With the trainees I'm teaching computer programming (cf. "Malawi's Mzuzu e-hub empowers youth through tech").

Starting with Humanity—Growth and Technology Otherwise Hurt You

Mon, 30 Oct 2023 00:00:00 +0000

In September 2023, I was reading Gabrielle Zevin's Tomorrow, and Tomorrow, and Tomorrow for the month's book club I have been organizing with my friends back in Canada for the last 12 months.

It's a story that uses digital games as an interface between human beings; throughout the book, the nerdy characters communicate through the games, grow with the games, and are helped by the games. Takeaway? Humanity is the foundation of the digital world. The book thoroughly illustrated the point by connecting the characters' attitudes to games with something more realistic, such as money, reputation, despair, hope, friendship, and love. As a millennial born and raised in Japan, who spent most of his teenage days playing video/computer games, and pursued a career in tech after being attracted by such digital content, the book instantly became special to me.

* Although I read the audiobook version, I coincidentally found a printed copy of the book at an Italian-owned restaurant in my neighborhood in Malawi.

Dilemma over "Best Practice": How We Could Develop Data Protection Practices in Malawi

Fri, 29 Sep 2023 00:00:00 +0000

Here in southeastern Africa, I see Malawi's digital transformation is still immature and yet to be caught up in many aspects. As a software developer volunteering in the country, one of the immediate questions I came up with was about data privacy. Unfortunately, while the use of digital technologies is slowly but certainly progressing in various sectors, data governance is rarely spoken as far as I can see. Let me think about what I might do if I were a Malawian service provider who lets users install an application to their phones, collects and stores data digitally for analytics purposes, or launches a web portal for businesses.

The basics

Importantly, as GDPR, CCPA, or any regional regulation define, it's not a matter of where you (developer; service provider) are physically present. It's more of who uses your service from where, and how their data is stored and processed. In that sense, we first need to be clear about the target audiences and why we collect their data for what purposes. Then, if the audiences do include people accessing the service from a certain country, that's where we should double-click their local regulations.

Definition, Role, and Current Status of Digital Literacy in Malawi #LiteracyDay

Fri, 08 Sep 2023 00:00:00 +0000

It's International Literacy Day. As an ICT Advisor at a Malawian local organization, I'd like to review the state of digital literacy in Malawi. An important context here is that, as the Digital 2023: Malawi reported, the country's internet penetration, and digital literacy inherently, stays very low, and the urgency let the Government of Malawi implement the Digital Malawi Program to accelerate the country's digital transformation.

But, what does the reality—an actual state of digitalization in the country—look like? What do all the data and development activities mean to Malawian's daily life and future? How digitally literate are they? What's digital literacy for them, after all?

TL;DR

In Malawi, access to the basic IT necessities such as computers, the internet, and smartphones is still very much under development, and hence it's not trivial for their young- and rural-centered population to talk about digital literacy.
Even if people do have access to digital tools, their knowledge and skills are usually basic and very limited, compared to my Western view of being digitally literate.
Businesses rarely talk about (and hardly know how to implement) cloud, data, security, or privacy. Clearly far from AI, Blockchain, or AR/VR. Their interests are similar to what I saw 10 years ago in Japan and the US, such as the use of smartphone applications and social media, or computer-driven mechanization in some sectors. The role of digital literacy in this context is to empower Malawians, especially young people, for the country's bright future, before things become too late.
That said, what being digitally literate means also depends on cultural traditions, work environment, and local regulations. It's still too early to make a judgment and say how it should be.

To begin with, we should better understand who are the "people" we are talking about, in terms of population statistics.

Starting Field Study on How Information Flows in Malawi

Wed, 23 Aug 2023 00:00:00 +0000

On July 28, 2023, I moved to Malawi, a beautiful landlocked country in southeastern Africa, for a Canadian international volunteering program at World University Service of Canada (WUSC). The program is funded by the Canadian government, and the organization sends Canadians, including permanent residents, to developing countries to advance Canada's Feminist International Assistance Policy.

As of writing, I've already spent three weeks in the country. Interestingly, there hasn't been much of a culture shock so far, probably because I visited mostly urban areas where we see a lot of influences from Western countries; at a surface level, if you subtract the economic growth from Southeast Asia, it's here.

* Lake Malawi, the source of fresh and delicious fishes like chambo and usipa.

However, the reality isn't that simple, and one of the poorest countries in the world is suffering from various forms of gaps, between rich and poor, urban and rural, and the other countries. These gaps are the very reason why I'm here, and you can read more about the motivation at my fundraising page.

Digital Transformation in Malawi

Tue, 01 Aug 2023 00:00:00 +0000

Scope

Led digital product and infrastructure strategy at a Malawian non-profit, from in-field user research and capacity assessment to mobile app delivery and web performance optimization. Managed a team of 10 communication and IT professionals as an Acting Director while representing the organization nationwide through conference presentations, workshop facilitation, and stakeholder engagements.

Technology

Domains: Education, Non-profit Operations, Digital Marketing
Platforms: Google Analytics, HubSpot, Power BI, Tableau
Implementation: Python, SQL, WordPress, JavaScript, .NET (MAUI), AWS, cPanel

Key Contributions

Strategic Guidance: Conducted a capability assessment across security, privacy, data, and AI domains, translating a 5-year organizational vision into an actionable digital roadmap with KPI tracking across communications, IT, and data priorities
Digital Marketing Implementation: Led analytics-driven content and engagement strategy for the organization's WordPress web presence, emergelivelihoods.org, achieving 8x year-over-year traffic growth while improving accessibility and frontend performance
Mobile App Delivery in Low-Resource Context: Refined UI/UX design, roadmap, and outsourcing strategy of an e-learning application, unblocking the app release in 3 months while achieving 80% cost reduction

Testimonial

Takuya has great work ethic and demonstrated strong professionalism whilst working with Emerge Livelihoods. He's always timely and committed to excellence in delivering diverse projects and services. Additionally, Takuya was great at planning and executing ideas as well as documentation through report writing and blog articles.

Runner's Search for Identity

Tue, 20 Jun 2023 00:00:00 +0000

Let's talk about running.

I've been a consistent runner for over five years now. To me, running is a way to have a sense of solitude. The quiet, focused moments naturally allow me to connect with my inner self and organize cluttered thoughts. The process of internalizing what I'm feeling/experiencing at that moment always helped me to maintain and strengthen my mental health however circumstances I was in (e.g., working hard on a big project, writing a research paper, and traveling around the world for business, or even on vacations).

Moreover, running is a liberating activity for those who seriously hate competing with others, like me; unless you are a professional athlete, the only "competitor" is yourself for the sport, and you can freely play the game however you want only with a decent pair of shoes. Last year, the at-my-own-pace consistency brought me to a point that I've envisioned for a long time—I finished my very first sub-4 marathon at the Royal Victoria Marathon in Oct 2022.

Information Sustainability, Mindful Consumption, and Healthy Engineering

Sun, 21 May 2023 00:00:00 +0000

It's easy to build an analogy between information and food: both of them are substances we consume daily, and unbalanced, too much consumption will result in serious physical, mental, and societal problems. Make sense, right?

Even so, the capitalist society is filled with a bulk of cheap, mass-produced, and processed products, which are very "tasty" thanks to corporations' engineering efforts. And human brains are designed in a way to be hooked by such strong attractive tastes¹. We can't stop craving salty, sugary, and fatty food. That's what journalist Michael Moss reported in Salt Sugar Fat, and to me, the story of food giants resembles a lot with tech giants in the ICT (Information and Communications Technology) industry as we can't stop scrolling through Instagram feeds, YouTube videos, and online shopping sites. Often there is no intention (autonomy²) in one's decision-making process, and we just follow what our brain responds to the stimuli.

But remember, ultra-processed, too-tasty, and toxic foods are made by companies prioritizing profit over our health. Thus, blaming McDonald's, KFC, Kraft, or Kellogg for your obesity won't be a smart move³. Instead, it is consumers who need to change with an intention as Maya Adam at Stanford advocates mindful eating: an active eating habit that lets us be conscious about what to eat and how much we know where they came from⁴. That is, our body is made of what we consumed, and every bite has certain consequences.

When We Lose Autonomy—Whose Life Are You Living?

Tue, 25 Apr 2023 00:00:00 +0000

Autonomy is important. Nobody wants to be enslaved or manipulated by other people¹. In the context of AI ethics, ensuring one's autonomy by proper algorithm/interface design (e.g., transparency, fairness) is one of the biggest challenges researchers and practitioners are tackling these days.

But let's be honest: how much of our value, identity, and behaviors are truly determined by myself? Imagine the most valuable thing in your life. Isn't your definition of success something given by the capitalist society? Are we surely making an informed, conscious choice when we buy and consume?

In other words, is this a life YOU chose? Do YOU love it?

Autonomy brings meaning to life

I keep coming back to these questions since I entered my 20s, and thankfully, I can confidently say yes as of now. Yet I still find it difficult to openly talk about the point with friends, family members, or those who I admire because our lives are experiential and situational; even if I'm deeply worried about someone's machine-like life, my view doesn't necessarily represent your view of the world². That's why autonomy is a sensitive topic, and there are endless philosophical and religious debates in the literature.

Three Perspectives on Large Language Models and Their Emerging Applications

Fri, 07 Apr 2023 00:00:00 +0000

As an engineer working on machine learning and data science for several years, now is indeed an interesting moment to see the impact of large language models (LLMs) such as OpenAI's GPT. Overall, I'm personally optimistic about the current situation and the near future, where a lot of simple work can be supported by the machines¹ (say ChatGPT, for example), and we can alternatively invest in the further possibilities unlocked by the applications. Yet it is not a complete substitute for human labor as the models' behavior is bounded by their training data and algorithms; that is, as long as our goal is to create something for humans to consume, rather than sitting back and watching machine-to-machine conversations, it is us who must understand the limitations, tweak the inputs, and assess the outputs.

Here, I'd like to share three perspectives I have built over the last few weeks thanks to the meaningful discussions I have had with my colleagues, friends, and mentees.

How Information Flows: From Field Studies to Risk Mitigation

Thu, 30 Mar 2023 00:00:00 +0000

*****: If you leave me free, I naturally flow Point A to Point B, but it is not always unidirectional; under a certain condition, I could go in the reverse direction or circulate in a surrounding environment. Ultimately, the movement depends very much on the other's behavior. We are mutually connected as part of larger systems, and our mission is to carry out something important for everyone to survive. Be careful—When you make me upset, I take your action seriously and do not hesitate to threaten your life. Who am I? Well, people call me information.

INFORMATION: I am fluid, a substance that has no fixed shape and yields easily to external pressure.

——This is how I perceive the concept of "information." It flows everywhere around the world, both digitally and physically. The volume is often overwhelming for us to digest (flooding; information overload), while it doesn't always stream equally to the branches (drought; digital divide). In extreme situations, these events can even be life-threatening.

The Complexity of Simplicity: Tacos, Sushi, and ChatGPT

Tue, 21 Feb 2023 00:00:00 +0000

When a particular product is presented in an extremely simplistic form, it has become harder for ordinary people to recognize the complexity behind it, making us unconscious about what's going on outside of our sight and triggering an over-simplified reaction.

From the end of 2022 to the beginning of 2023, I traveled from the US to Mexico to Japan.

One thing that kept sticking to my mind throughout the trip was about invisible factors turning the complex ingredients into a minimalistic deliverable. In his beautiful writing of Why Information Grows, physicist and scholar César A. Hidalgo defined product as a crystallized form of information. So, when we face crystallized information, how can we perceive the actual complexity embodied in the objects? In other words, how creators who crystallize the instances can help their audiences to better look-n-feel the details of their work? Or, is it an unnecessary consideration at all?

Think of tacos as a sample category of "product." At my first travel destination, Austin, Texas, my friend invited me to explore some restaurants where we find the local's favorite Tex-Mex tacos¹. To me, these tacos were obviously complex; there are a lot of flavors and toppings on tortillas, and it has been presented more as a complete meal rather than casual street food:

My 2022 Annual Review: Towards the Deepness

Sun, 18 Dec 2022 00:00:00 +0000

A single word describing my 2022 is connection.

To overcome day-to-day struggles and establish fresh life as self-employed, as well as permanent resident of Canada, I was continuously supported a lot by new and old human connections I, fortunately, have had across the globe.
Through physical experiences like traveling, hiking, and kayaking, I deeply realized the interconnection between myself and the earth, which embodies complex history and natural environment.
The dots from my nearly-a-decade-long learning journey in data mining, complex systems, and internet ethics have been linked to each other, and they surfaced actionable insights about locality and humanity in tech.

As always, I would like to thank everyone who interacted with me on various occasions. Every single conversation with any of you brought new perspectives to me, and regardless of whether it's positive or negative, I, as a dividual, am defined by you. In other words, the 2022 version of myself cannot exist without others.

Connecting to myself

Speaking of connection, I also learned how to connect with myself. Working more than 40 hours a week, being exhausted, yet feeling (unnecessary) pressure to work harder—These are NOT normal at all. We should realize the weirdness. The idea that rest is the form of resistance against capitalism came from a book Rest Is Resistance; the societal system enslaves us to be machine-like in a competitive, never-ending journey of pursuing hyper-productivity.

Mentoring on @ADPList. Here's What I Think. #ADPListWrapped

Thu, 08 Dec 2022 00:00:00 +0000

After being independent, several people encouraged me to share my experiences with others as a mentor, since some of the backgrounds are unique yet relevant enough to those who are working inside/outside of the industry:

Took multiple roles between engineering, science, and business;
Served different scales of business at start-up, Big Tech, and as an independent;
Worked globally (both physically and remotely) on diverse verticals such as e-commerce, gaming, online media, and supply chain.

Following the comments, I registered to ADPList, a free online mentoring platform, and mentored about 10 people over the last three weeks.

Note that everyone's life is highly contextualized in a certain situation, and mine is not an exception. Therefore, a general recommendation is NOT to take a single person's advice too seriously and try to incorporate multiple perspectives from diverse people.

Would you be willing to take an action NOW?

First and foremost, regardless of the form of mentoring, I ultimately give only one take-home message to mentees: Your specialty comes only from experience. The point my mentor made explicit more than five years ago is the most important (and only) statement I still recall regularly.

Data Are Created, Collected, and Processed by People

Sun, 04 Dec 2022 00:00:00 +0000

That is, information in human society is localized, and there is always the existence of real humans throughout the data lifecycle from their generation to collection to aggregation. Even though a specific dataset you are analyzing looks simple, you cannot (and must not) reach a conclusion without understanding its context—how the data are generated, collected, and used by whom, and to whom your work is presented.

Why people matter in data science

Data Feminism is a new way of looking at data with a special emphasis on underlying human biases. Unlike what we imagine from the wording, data feminism is not just about gender inequity behind data science, though it's certainly one of the most representative examples. In a wider sense, the idea reminds us that human biases exist everywhere across the data pipelines. Thus, every practitioner needs to acknowledge the facts and turn the understanding into conscious actions whenever they interact with data.

Fluid People and Blended Society: How Systems Model "Dividuals"

Sun, 27 Nov 2022 00:00:00 +0000

"Let $\mathcal{U}$ be a set of users ..."

User modeling research often begins by stating a set of individuals to formulate. But, since our world is highly contextualized as economics and complexity science have revealed, representing a user as an element of a logical set may be an oversimplification. Here, the fluidity of user behavior explains the importance and potential risks of recent trends in personalization systems research, such as context-aware, group, multi-stakeholder, and/or serendipitous/diversified recommenders.

There is no such a thing as an "individual"

Most importantly, a single user can take multiple different states. The term individual means indivisible, meaning a single person is the smallest unit in human society. However, in reality, each of us consists of multiple identities, and hence the unit is actually divisible into smaller chunks, resulting in what anthropologists call dividual.

The concept of dividual represents the contextualized nature of our behaviors. For example, I can be divided into multiple units depending on a situation, and I behave differently for each:

The Locality of Information and Technology

Sat, 19 Nov 2022 00:00:00 +0000

Information, technology, and inherently everyone's life are highly contextualized and localized. Understanding the simple fact would bring a dramatic shift in how we see the world. In particular, reality tells us the importance of characterizing our world without suppressing the identity of its elements.

The driving force behind the world

Most importantly, there is nothing like a "united" world, and the global society is rather a set of different worlds each country, city, organization, and person holds. I consider myself a global citizen so I don't overlook the fact that we all are interconnected in some sense on this beautiful planet, but it doesn't necessarily mean the world is united at the planet scale in the political, geographical, ethnological, and/or environmental sense.

When I fly to multiple cities across the globe, it's clear that none of them is the same, even if I focus only on a single aspect: carbon footprint.

For instance, in their awakening book Blockchain Chicken Farm: And Other Stories of Tech in China's Countryside, Xiaowei Wang showed that the uniqueness of rural China comes from the complex cultural and political background, and the situation could differ among villages and communities. Similarly, consider your home town and country. It won't take so long to realize the shape of a greater society is derived from multiplications (not summation) of individuals' smaller worlds. That is, other person's view (i.e., "resolution") of the world won't be the same as mine, and it's not a matter of technology, the hardware consisting of our daily life.

Google Cloud Dataflow and Its Positive Impact on Developer Productivity

Fri, 11 Nov 2022 00:00:00 +0000

With the specialization on Coursera, I've been on a not-too-fancy yet meaningful learning curve over the last few months in the context of machine learning on Google Cloud.

One obvious challenge in cloud data management and machine learning is the need for integrating multiple data sources and processing massive records in multi-phases without loss of visibility, scalability, and maintainability. Here, I found Google Cloud Dataflow, or Apache Beam as its foundation, is particularly promising because the hosted Apache Beam-based data pipeline enables developers to simplify how to represent an end-to-end data lifecycle while taking advantage of GCP's flexibility in autoscaling, scheduling, and pricing.

Here I list my first impressions after spending a meaningful amount of time using Dataflow in some projects.

Programming model as an implicit enforcement mechanism

First of all, I like the programming model as an engineer. Like Apache Airflow's >>, a "good" data engineering tool offers a unique, simplified programming model (syntax) to make data pipelines not just programmable but accessible (i.e., readable and modifiable) for every team member. To give an example, the following code snippet is an official example of counting words from texts:

Connecting the Dots in Complexity

Thu, 03 Nov 2022 00:00:00 +0000

At a certain point in my university days in Japan, I was so into reading Complexity: A Guided Tour written by Dr. Melanie Mitchell, Davis Professor at the Santa Fe Institute. I even took their online course (putting the other textbooks needed for the university's exam prep aside).

Why? I wanted to find out how society on the internet evolves. As information technology becomes the norm among early- to late-majorities, the atmosphere on the internet had clearly shifted from digital "nerdy" communication hubs, which I deeply loved, to competitive marketplaces where people seek profit and reputation for practical reasons. I was curious about why/how the transition happened so rapidly, almost like an uncontrollable natural phenomenon. Eventually, I learned such a network structure is known as a complex system and then started working on social network analysis for making sense of the complexity. I've gotten a handful of actionable insights and encountered a larger set of more complicated problems ever since then.

About 10 years later, I recently attended an art class hosted by a local art gallery and artist in Canada. We practiced meditating in the forest, maximizing the sensitivity of our body, and turning perception into expression.

Updating My View of Nuclear Power

Thu, 27 Oct 2022 00:00:00 +0000

I am concerned about climate change, and what's clear to me after studying several resources is that we cannot discuss the planet's future without thinking about how quickly and safely human society can adapt to using nuclear power at scale. That is, the speed of climate change is just too urgent to rely fully on the other actions we can think of, such as improving consciousness of CO2 emissions, understanding technological advances and adapting our consumption patterns (e.g., buying an EV car, eating less beef), accelerating government-led, collaborative effort on making a city greener, and rethinking a driving force of the capitalistic world. All of them are effective and important, but a key takeaway is that we'd need to prioritize them wisely and establish the most aggressive portfolio given the urgency.

Challenges

Talking about nuclear power is not easy (and is often considered a sensitive topic as it's strongly tied to political stances and directly relevant to our health). In my case, as a Japanese person who closely experienced the 2011 Fukushima "disaster"¹, I still spend a hard time making a fair assessment of the technology even though people commonly make optimistic statements, including but not limited to Bill Gates and the author of a New York Times best-selling climate book, as well as some of my climate-conscious friends.

Why We "Productize"

Thu, 20 Oct 2022 00:00:00 +0000

Assume you are a business owner already providing some solutions to the users.

Should I consider making significant growth from now on?
If I should, what is the best path forward for growth?

In that situation, productization can be an essential idea to dive deep into. Importantly, "product" is not a one-size-fits-all concept, and there are multiple, gradual paths to productize an existing solution in practice; it is not a binary state of a business offering. Furthermore, I believe not-to-productize (i.e., eliminating a sense of business and focusing on humanity) can also be an option depending on your answer to the first question.

Gradient of offering

A single tech-driven solution in the industry, which is commonly referred to as a service or product, undergoes different stages depending on the organizational scale, business model, and maturity of the solution. Thus, to determine the next steps, it's crucial for businesses to finely define a solution by its stage, understand which stage they are currently at, what the desired state looks like, and how to effectively move from one stage to another.

Ethics in Recommendation Pipeline—A First Look at RecSys 2022 Papers

Fri, 14 Oct 2022 00:00:00 +0000

RecSys 2022, the 2022 edition of a top academic conference in the field of recommender systems, happened last month in Seattle. Even though I didn't attend the event like the last couple of years, it's still fun to check out the papers and see how the research trends, as well as the industry's focuses, are shifting.

At a glance, while many studies continuously try to fulfill the gap between theory and practice by applying state-of-the-art neural network-based techniques to their own problem spaces (e.g., news, music), more research also started casting "deeper" problem statements that cannot be easily addressed by the conventional optimization problems: privacy, fairness, and offline evaluation vs. online performance. In fact, these topics are not new, and we've seen the presence of relevant work over the last few years as I summarized the 2021 trend as user-centricity. However, I feel the number of papers incorporating these perspectives into a key message has increased at a steady pace.

In my view, such a trend surfaces an underlying challenge about ethics in recommender systems. As Big Tech popularized the power of personalization in their services, we see not only the benefits of these capabilities but a flip side of them such as privacy threats, polarization, and loss of autonomy at large. To address these concerns, developers need to make conscious design choices at every step of recommender system development, ranging across UI/UX design, data collection, model training, evaluation, deployment, and monitoring.

My First Month as Self-Employed

Wed, 05 Oct 2022 00:00:00 +0000

It's been a month since I officially became self-employed in Canada. So far, things are going quite well, especially in terms of quality of life. Time flows slower, and a day is surprisingly longer than I used to feel. Even though my monthly income in Sep 2022 was roughly 50% of the amount I earned until the previous month, I feel the "cost" for a better life is pretty cheap. And thankfully, the 50% amount is still not a threatening number for a single person living in the city.

That said, I know it's too early to conclude; being in a situation where every minute literally counts needs a radical shift in how to use my own time, and I'm tuning myself daily to be a better version of myself in various ways. Here, let me jot down some of the key changes, observations, and tips I have had for the first 30 days.

What are you doing now?

You can find the answer in takuti.me/now:

Reviewing Things I Do Not Know About the Indian Act #NDTR

Thu, 29 Sep 2022 00:00:00 +0000

September 30 is the National Day for Truth and Reconciliation (hashtag: #NDTR) in Canada, a day that we look back on the dark history of the residential school system European settlers had implemented, raise awareness of its impacts, and take an action to rebuild the relationship with Indigenous Peoples.

As a foreign national living in Canada, I see understanding the local history, especially the ones about Indigenous Peoples, is an important mission, and I've been continuing the journey through learning Canada's history and taking an online course. Here, my recent reading of 21 Things You May Not Know About the Indian Act: Helping Canadians Make Reconciliation with Indigenous Peoples a Reality was a highly insightful and great use of time for the week of Truth and Reconciliation.

The book gives readers a succinct collection of important facts and references about how terrible the past & present is, and what we can do as an individual. Importantly, the content is NOT about the history that's already over, and the impacts rather still visibly exist in various scenes.

Why Your Job Title Matters (Cont.)─Technology for the People

Thu, 22 Sep 2022 00:00:00 +0000

As I previously wrote in Why Your Job Title Matters, Quality Habit-Formation, and Ethical Product Developer, I've been intentionally calling myself as Product Developer rather than obvious titles such as Software Engineer, Data Scientist, or Machine Learning Engineer. Now, since I've become a freelancer, I have greater flexibility in choosing how I advertise myself on the internet, and I want to question whether Product Developer is truly the right way to describe what I (would like to) do. In particular, this article double-clicks the use of the term "product".

Spoiler alert: As you can see on takuti.me or social networks, I'm temporarily changing my bio to Freelance Software Developer.

What makes "product" so weird

Most importantly, the term "product" can deeply imply a sense of business. In my definition, one of the biggest differences between a piece of random software and a matured product is whether a deliverable is considered to be valuable in the capitalistic world. For example, even though I have more than a hundred GitHub repositories (i.e., pieces of software), it's hard to categorize them as a "product" compared to the quality and economic impact of the production code that I had previously written in a company. But, does that mean all personal projects are meaningless, or relatively less valuable, at least? Common perception would be YES, which I want to challenge as an individual.

Seeing Past and Present in Coursera "Machine Learning on Google Cloud" Specialization

Wed, 14 Sep 2022 00:00:00 +0000

I have recently got a chance to review machine learning (ML) basics through Machine Learning on Google Cloud Specialization on Coursera. Although none of the contents was really new to me, it was a well-developed series of courses that gives us a high-level overview of the field both in theory and practice, accompanied by hands-on assignments on Google Cloud so we can familiarize ourselves with how to turn the basic concepts into workable implementations. I would definitely recommend the course if someone has a plan to build a data/ML pipeline on Google Cloud from scratch knowing little to nothing about ML.

The basics

What you will learn from the specialization can be summarized as follows.

How Google does Machine Learning
- Introduction to Google's ML offerings, such as Vertex AI and BigQuery ML.
- Showcasing best practices for solving a real-world problem by using data & ML, with a particular emphasis on explainable and responsible AI.
Launching into Machine Learning
- Understanding the basic steps in a common ML lifecycle such as hyperparameter tuning, evaluation, and deployment.
- Implementing solutions with Vertex AI Workbench (Jupyter Notebooks) and BigQuery ML by taking full advantage of AutoML.
TensorFlow on Google Cloud
- Creating non-AutoML, deeper neural network-based models with TensorFlow & Keras, and containerizing the resulting models.
- Diving deep into key ML concepts, like loss function and regularization.
Feature Engineering
- Highlighting the importance of feature engineering and some of the common techniques.
- Engineering features on BigQuery ML, Keras, and Apache Beam-based Dataflow, so that we can "package" a featurization pipeline and manage its resulting feature set in Vertex AI Feature Store.
Machine Learning in the Enterprise
- Seeing how a cross-functional team can work on Google Cloud across multiple services, not only for scalable model training and optimization but for data/model management and governance.
- For instance, we manipulated raw data stored in BigQuery with Dataprep by Trifacta, which reminds me of a conversation from Trifacta with Joe Hellerstein - Software Engineering Daily.

Note that the majority of the assignments are based on a collection of Jupyter notebooks available on GitHub.

How I Define "Artificial Intelligence"

Fri, 09 Sep 2022 00:00:00 +0000

My recent reading of the novel "Klara and the Sun" written by the Novel Prize-winning writer was lovely. Beyond the simple joy of reading the story, the book enabled me to rethink the concept of artificial intelligence "AI" from an ethical standpoint.

In my opinion, what's unique about Klara and the Sun was how the author illustrated an AI, Klara, as a character. What I mean by that is how realistically the story depicts the emotion and behavior of the "machine," and I think it is one of the few differentiators we can see among numerous Sci-Fi books/movies that set an AI as a core of its storyline, like A.I. and Her.

In other words, while Klara and the Sun is not outstanding for me in terms of the underlying messages the story tries to convey, like "love", "relationship", and "beauty of the human world" that many Sci-Fi stories commonly speak, the author beautifully illustrated interactions between humans (e.g., Josie, Rick, Mother) and AI (Klara) with detailed psychological description both implicitly and explicitly.

Privacy Policy

Thu, 08 Sep 2022 00:00:00 +0000

This website, takuti.me, uses Google Analytics to allow tracking technologies through the use of cookies, to, among the other things, analyze and track users' use of the website, determine the popularity of certain content, better understand online activity. By accessing the website, you consent to the collection and use of your information by the third-party tool. You are encouraged to review their privacy policy such as Google Analytics Terms of Service and Google Privacy & Terms, and contact them directly for responses to your questions.

This website does not collect and transfer personal information other than anonymized usage data. However, if you do not want any information to be collected and used by tracking technologies, you can opt-out by disabling cookies on your browser, contacting the third-party vendor, and/or visiting Network Advertising Initiative Opt-Out Tool or Digital Advertising Alliance Opt-Out Tool.

Paddling in the Life, and Currently I'm a Freelance Software Developer

Fri, 02 Sep 2022 00:00:00 +0000

I quit my software engineering job at Amazon on August 31, 2022. One week before my resignation, I was in the ocean for a week-long kayaking trip with my friends on the west coast of Vancouver Island, which turned out to be my longest adventure ever in such a remote environment without any utilities.

The vacation was unbelievably fantastic not only because of the unreal scenery & contact with wildlife but also as an opportunity to spend an abundant amount of time on self-reflection; I felt so good staying disconnected from the internet for an entire week, and I realized that having time to kill with an empty mind is the richest thing we will rarely experience in the modern life.

Takeaways? Life is short, and it can be like a paddling trip.

Unpredictable, like the waves and tide.
Absolute silence on the islands — Quality of life, in a true sense.
Feeling of being part of the earth — How tiny our day-to-day struggles are.

Accumulating unpredictable experiences

Life is unpredictable, and my journey moving from one place to another on a case-by-case basis is pretty much like paddling in the ocean. While I can cruise at a comfortable pace with minimum use of power under perfect conditions, strong wind and bigger waves sometimes disable me to proceed to the desired direction in the shortest path, where I need to save energy and wisely use the pieces of equipment to get things right.

Dilemma of Creative Selection

Fri, 12 Aug 2022 00:00:00 +0000

"Creative Selection: Inside Apple's Design Process During the Golden Age of Steve Jobs," a book written by an ex-Apple software engineer discussing how to make a successful product like iPhone, was fun to read.

In short, the essence of building such a great product is making a team really small and focused. The early days of Apple had a strong culture of valuing rapid demo-improvement iterations, which enable a product to undergo a "creative selection" process; like the natural selection process, rapid iterations allow the team to adaptively select the superior design of software/hardware systems and evolve the product to the right direction.

Great Team, Great Product

To be more precise, the author highlighted there are 7 essential elements of Apple's product development.

Inspiration: Thinking big, and imagining what might be possible.
Collaboration: Working together with others well and seeking complementary strength.
Craft: Applying skills to achieve high-quality results striving to do "better". An example is continuous performance testing when they developed the original Safari browser.
Diligence: Never shortcut, and do necessary grunt work.
Decisiveness: Making tough choices and refusing to delay.
Taste: Developing a refined sense of judgment and finding the balance that produces a pleasing and integrated whole
Empathy: Trying to see the world from other people's perspectives and creating work that fits into their lives and adapts to their needs.

My favorite essence is indeed empathy. In any case, all the points eventually direct us to the following core idea—The continuous demo-improvement cycle run by a small, focused team triggers the selection process and pushes a boundary an original idea might put.

Recommendation.jl Came Back to #JuliaCon 2022

Sat, 06 Aug 2022 00:00:00 +0000

At JuliaCon 2022 @ Online held during the last week of July, I gave a lightning talk about Recommendation.jl, a Julia package for building recommender systems. It's been 3 years since the last time I talked about the package at JuliaCon 2019, and, since polishing the (outdated) implementation towards v1.0.0 is one of my recent focus areas, I decided to showcase the updates of the package and take it as an opportunity to review the remaining steps ahead of me.

Check out the presentation at YouTube:

Over the last few months, my biggest technological interest has been in what defines the "goodness" of data-driven applications, including recommender systems. Intuitively, more accurate prediction is better as algorithmic recommendation eventually encourages the users to "efficiently" use their time on the applications. However, it often causes unintended consequences as we've discussed in the context of ethical product development, data science ethics, and humane use of technology. Thus, I do believe non-accuracy aspects of the systems are equally or even more important, and I'm glad that I was able to turn the idea into actual implementation as part of the Julia package.

Indigenous Canada: Storytelling, Community, and Sustainability

Sat, 11 Jun 2022 00:00:00 +0000

It's National Indigenous History Month in Canada, and I just finished University of Alberta's 12-week course of Indigenous Canada.

When you visit Canada, you can easily find a connection to Indigenous culture in many forms such as road signs and public arts. However, it is not easy for everyone to understand the complex history behind the scene; if we take the cultural icons without knowing the background, we cannot establish a mutual understanding between Indigenous and non-Indigenous peoples in a true sense, and it's impossible for the society as a whole to build a better future in this beautiful country of Canada. Here, the online course was a great introduction to Canadian history and gave me an opportunity to realize how little I know about the country, where I currently (and permanently) live.

In fact, every single fact I learned from the course was eye-opening, but there are 3 key takeaways I strongly recall:

For Indigenous people, storytelling is a major way to transfer knowledge from generation to generation.
Community-based activity is vital for Indigenous people to live, learn, act, and strengthen their identity.
Indigenous lifestyle is based on deep appreciation for our mother's land and natural resources, which can be essential for modern sustainability development.

It should be noticed that these Indigenous ways of living/thinking overlap with what's been discussed lately in the context of global economy, and the practices have been completely normal for Indigenous people from the beginning. That is, Indigenous history, in general, is important not only for correctly understanding the past but also for making present & future better.

Becoming Permanent Resident of Canada

Sun, 05 Jun 2022 00:00:00 +0000

It's been more than a year since I relocated to Canada in Feb 2021, and I have officially become a permanent resident of Canada this week. My major motivation of being a permanent resident is to maximize the flexibility of choosing a job and possibly getting education in this country.

Thanks to BC Provincial Nominee Program - Tech (BC PNP Tech), the application process was very smooth even though I changed an employer in the middle; I was fortunately able to work with highly supportive attorneys both in the previous and current company, and my PR application has been approved much quicker than I expected¹. Here is a rough timeline I have undergone:

(2021/02) Moved to Canada from Japan with a work permit. Took IELTS, an English test required for PR application, 2 days before the flight.
(2021/06) Applied for BC PNP Tech.
(2021/07) Accepted BC PNP nomination, and received an invitation to apply for permanent residence.
(2021/08) Changed an employer. Initiated BC PNP post-nomination process.
(2021/12) Re-received BC PNP nomination and invitation to apply for permanent residence.
(2022/02) Applied for permanent residence.
(2022/06) Received an approval of my PR application, and the status has been confirmed in less than a week.

Note that the processing time varies depending on various factors. Everyone must refer to the government's website for the latest information. Meanwhile, COVID-19 impacted the process both in positive and negative ways, but I believe none of them was a significant blocker/accelerator. On a positive side, for example, the university I graduated from temporarily accepted a request for issuing a certificate of degree by email, which usually can be requested only by mail with a paper-based application. On the other hand, there was a little bit of confusion when I was trying to obtain a police certificate; the police office's fingerprinting service and embassy's operations were irregular during the pandemic.

My First Post-Pandemic International Trip

Sat, 04 Jun 2022 00:00:00 +0000

Last week, I was traveling to the US to meet some of my friends, which was my first post-pandemic international trip¹. It was simply awesome, and the experience has strongly amplified my traveler identity. Even though I already restarted enjoying domestic travels, international ones are still a lot different in terms of the experience we can gain and people we can meet with. Speaking of travel restrictions, as of writing, entering the US by air requires a pre-flight COVID test regardless of vaccination status, whereas entering Canada has no restriction as long as you are fully vaccinated.

Here is what I thought throughout the trip, so I don't lose the fresh feeling.

Importance of real human interaction. I'm fortunate enough as I have several good friends in the US who give me a warm welcome, and the past week was full of face-to-face communications thanks to all of them. During the pandemic, we realized how valuable it is, and we finally became able to make that happen again. In particular, since I'm psychologically in a tough situation lately, every single moment meeting with someone I like/respect was literally a relief. I'll definitely come back more often because one week was too short to meet with everyone I wanted to talk to.

Rethinking Why, When, and How I Learn

Mon, 23 May 2022 00:00:00 +0000

Freakonomics Radio's recent Goes Back To School series was eye-opening as the episodes nicely highlighted how top-tier U.S. universities are operated like "business" by trying to differentiate their products to win the market while preserving their rarity/status in the society based on the strict selection process. As the education systems are commoditized in the developed countries, the real value of higher education and the definition of "good" education have become ambiguous. Thus, I personally believe we, as a learner, need to pause for a moment and rethink why, when, and how we learn as an individual. Otherwise, we likely end up with losing control of own life and being one of numerous consumers who is unknowingly manipulated by the external forces—Zombies.

Why I learn. For me, it's all about curiosity. For example, while I started learning software engineering when I was in high school and continued the journey ever since then, I don't see the skillset as a means to secure a job or income source; I wouldn't be surprised if I'm in a completely different occupation next year in case my curiosity is run out. In fact, my focus is leaning more towards nature, history, and social science lately so I can equip myself to tackle real problems I am personally experiencing on a daily basis. Hence, the next job may or may not be completely irrelevant to what I learned over the last decade, and making such unrealistic possibilities happen is the biggest outcome learning will eventually bring.

Am I Zombie? Autonomy vs. Recommendations on the Internet

Sun, 15 May 2022 00:00:00 +0000

Nowadays, our day-to-day decisions rely too much on how others think. The others are not from a close relationship like family, friends, and colleagues. In fact, the people who are making an influence on our behavior are complete strangers on the internet.

Someone gave a 5-star review for this product.
Why don't you go to this place where many people like you enjoyed it?
By the way, how about this item you might overlook?

As the technology advances, online review and algorithmic recommendation become more accurate and make our life more "efficient". Consequently, it is very difficult for us today to enter a really bad restaurant, read a horrible book, or listen to a song that is completely out of your taste.

But, wait—Do we really need that degree of certainty every day at every single moment?

Of course, the information is good for specific situations when you must not make a bad choice (e.g., business dinner, gift for a loved one) and/or you intentionally want to accomplish the task with minimum effort on a regular basis (e.g., grocery shopping, where you don't need any big surprise). However, at the same time, I believe relying too much on online recommendations sacrifices our humanity and decreases the quality of life at the end of the day.

What I Like About Microsoft Recommenders Repository

Sat, 07 May 2022 00:00:00 +0000

I recently found Microsoft's Recommenders repository is particularly useful to understand common discussion points when it comes to recommender systems. The motivation and brief history of the repository can be found in their paper "Microsoft Recommenders: Tools to Accelerate Developing Recommender Systems," which were demonstrated at RecSys 2019 and WWW 2020.

What I like about the repository can be three fold:

High-quality, well-written Jupyter notebooks
Minimal functionality on its PyPI package
Consideration about non-accuracy metrics

High-quality, well-written Jupyter notebooks. Even though the repository contains an installable PyPI package recommenders (!), the most important part is a collection of well-written Jupyter notebooks that enable us to understand how to build recommender systems from data preparation and model training to evaluation and deployment.

Source: recommenders/examples at main · microsoft/recommenders · GitHub

Importantly, the notebooks are not just for a series of code snippets + inline comments (like most of the repositories do) but for providing detailed texts/references so we can use the contents as "tutorial." Moreover, as mentioned in the paper, integration tests use papermill for validating the notebooks.

Starting with Empathy to #GetReal, Beat Loneliness, and Be Ethical. #MentalHealthWeek

Thu, 05 May 2022 00:00:00 +0000

It's Mental Health Week in Canada. The theme empathy is a big topic for me as I recently made a dramatic shift in my focus as part of the latest update on my NOW page; currently, I have a strong feeling that being empathetic can be foundation of everything, including happiness in personal life, healthy mutual relationship with others, and sense of satisfaction on a job.

This spring, my physical and mental condition was the worst over the last few years for some reasons, and the experience gave me an opportunity to deeply appreciate the power of empathy that people shared with me. I'd like to leverage this Mental Health Week to look back on the experience. A key takeaway is that we all are extremely fragile, and paying attention to the things and people around us is a great way to complement the vulnerability in a collective way. Individuals are surprisingly powerless, and nobody's life is "solo" in a true sense.

Learning Canada's History

Sun, 01 May 2022 00:00:00 +0000

Canadian friend: "Do you know John Macdonald?"
Me: "I don't know. Who is it?"

Until a few months ago, I knew little to nothing about Canada's history, but I started feeling it's very rude and disrespectful. As a foreign immigrant, I personally think I should respect the culture, history, and people I'm regularly interacting with in this beautiful country. Hence, I have been intentionally building my knowledge through several museum visits and online resources (e.g., Indigenous Canada from University of Alberta on Coursera, a webpage from the Government of Canada: Discover Canada - Canada's History).

From my point of view, there are several key moments I commonly see across different resources:

1604-1608: European's settlement began.
1867: The Dominion of Canada was officially born.
1941: Japan attacked Pearl Harbor.
1996: Closure of the last residential school triggered the following reconciliation acts.
2015: The Truth and Reconciliation Commission's final report was accepted by the Government of Canada.

Understanding the "dark" history for Indigenous people. We cannot speak about Canada's history without mentioning the history of Indigenous. While I'm still at week 6 of Indigenous Canada from University of Alberta on Coursera, I've been shocked every week as I learn how selfish the European settlers were and how Indigenous's life, culture, and rights were threatened between the 17th and 20th century. The problem is actually very complicated as both sides had different ideas of diplomatic relations; though it's completely unacceptable, it is reasonably explainable why Europeans in the 17th century took such actions and why their relationship with Indigenous communities, including establishment of treaties and regulations, didn't go well.

"Why Do We Build This?" Humane Technologist's View of Bad Product/Project

Thu, 21 Apr 2022 00:00:00 +0000

I, as an ethical product developer, consider a product or project is "bad" when any of the following conditions are met:

End user is physically absent from the design process.
Success is measured purely by business metrics.
Product requirements are partially or entirely "given" by external forces (e.g., "leadership decision", "loyal customer X").

Over the last couple of years, I keep questioning myself as "Am I doing the right thing?" and reviewing where I am based on the criteria. However, in practice, focusing on those essentials is not easy to survive in the modern tech industry; the companies are still under a strong pressure of infinite economic growth in the world of capitalism, and pausing for a moment with these questions is simply "inefficient" as rapid iteration is a king for them.

Here, my recent learning from Foundations of Humane Technology gives me confidence that the struggle for "good" product development actually matters A LOT, and I feel I'm not alone in such a big problem space. I strongly encourage everyone to take the course. It should be noticed that my definition of ethical product developer is mostly aligned with what the course calls humane technologist.

Loneliness Is Worse Than Smoking, Alcoholic, Obesity

Sat, 09 Apr 2022 00:00:00 +0000

After reviewing several scientific evidence showing the fact, I'm urgently seeking a way to break my loneliness. To my friends, former/current colleagues, or random strangers, let's (re)connect.

Surprisingly, it is proven that the negative impact of social isolation is not only on mental health but physical condition and longevity in general.

NOTE: Loneliness and social isolation are different in a precise sense, but this article uses them almost interchangeably; we assume a person feels loneliness and is socially isolated with limited human relationships, which corresponds to my current situation.

Statistically speaking, smoking, alcoholic, and/or obesity are bad. No doubt. However, at the same time, not everyone dies due to these trivial reasons; there are many "unknowns" of which factor contributes to our death. Here, scientific research has revealed that a good chunk of health issues can be associated with loneliness or the lack of social relationships:

Social Relationships and Mortality Risk: A Meta-analytic Review in PLoS Med (2010)
Myeloid differentiation architecture of leukocyte transcriptome dynamics in perceived social isolation in PNAS (2015)

I originally found these studies on Nir Eyal's book entitled Indistractable, and we can partially read the particular section at: Happiness Hack: This One Ritual Made Me Much Happier. The book itself is simply great, and we see many practical techniques for quality habit formation based on strong scientific evidence the author previously introduced in another eye-opening book, Hooked. But eventually, the single statement kept sticking to my mind—"The clearest message that we get from this 75-year study is this: Good relationships keep us happier and healthier. Period." quoting Robert Waldinger's TED talk about "The Good Life".

Cross Validation for Recommender Systems in Julia

Sun, 03 Apr 2022 00:00:00 +0000

When it comes to building data-driven applications, evaluation matters. It matters A LOT, and an evaluation process is not always the same as how we calculate accuracy in the machine learning context (cf. Recommender Diversity is NOT Inversion of Similarity). Moreover, commonly speaking, online evaluation is the king to capture the unique characteristics of real-world events.

That said, it is still important to learn and leverage the basic offline evaluation techniques, so that developers can ensure a certain degree of validity of the implementation before production deployment. Thus, let's review what cross validation means for top-k item recommendation, and how the evaluation framework can be implemented in Julia, as part Recommendation.jl.

Assume we are trying to build and assess a top-k recommender based on a series of observed user-item events (e.g., click, purchase, rating). Here, a common approach is:

splitting the entire dataset into two sub groups, like 80% of training and 20% of "ground truth" testing events;
building a recommendation model based on the training data;
generating top-k recommendations to users, and measuring the quality of recommendation in comparison with the test data.

3 Pillars of Ethics' Scope: Society, Personal Relationship, and Individual

Mon, 28 Mar 2022 00:00:00 +0000

As Key Takeaways from UMich's Data Science Ethics Course told us, ethics is defined by social consensus. Here, another introductory book for ethics pointed out "society" is not the only problem space ethics considers; "how the right and wrong are defined by whom" varies depending on the size of relationship: Society, personal relationship, and individual.

First and foremost, the entire society is indeed the largest scope of ethics. We implicitly made consensus about what's good or bad, and the regulations such as the law and politics are determined by the consensus. Why is a crime bad? Yes, it's bad because the law disallows such a behavior, but more importantly, the law itself is based on social consensus; crime is bad because it's socially unacceptable. This is what the textbook calls ethics for social justice.

Meanwhile, we commonly face non-trivial "What should I do?" situation when it comes to more personal matters. To give an example, if there was an issue between you and your partner, I as a random stranger don't know what the right thing you should do; I probably say "It depends," and the right choice must be identified by the subjects based on their own consensus. This type of ethics that relies on smaller personal relationship is referred as ethics for love. Partner, friend, family—They are required to establish and follow their own rules to stay the precious relationship ethical.

It "Was" Ethical: Key Takeaways from UMich's Data Science Ethics Course

Sun, 20 Mar 2022 00:00:00 +0000

One of the most important takeaways from UMich's "Data Science Ethics" on Coursera is that ethics is defined by social consensus.

"What's good" changes

First of all, since the definition of "right" and "wrong" changes as time goes by & technology advances, we as a data science practitioner should keep questioning ourselves like "Is this socially acceptable?" throughout the data-driven product development lifecycle. In particular, by balancing an individual's value and public benefit, our deliverables must meet shared expectations from the society.

To give an example, we commonly see cameras at supermarkets or shopping malls nowadays, and many of us do not think our privacy is threatened because of the devices; there is social consensus that these cameras are "good" for security reasons and do contribute to making the public places safer. However, if the videos or images captured by the cameras are used for unintentional purposes, it becomes questionable whether the practice is ethical or not.

Thus, a boundary between good and bad tends to be fuzzy, and social acceptance criteria depend largely on the contexts when it comes to technology/data-driven solutions. Even though a behavior was ethical 10 years ago, it's possible that the same action is considered as unethical by the latest society. This is how ethics differ from religion, law, policy, and regulation, which are typically more stable and don't fluctuate that much. These fixed criteria rely on ethical behaviors defined by the society as of their original publication though; normally, society defines ethics, and regulation follows (after a long time).

Security, Privacy, and Ethics in the Web 3.0 Era

Sun, 13 Mar 2022 00:00:00 +0000

Web 3.0, which commonly refers to the technological trends around blockchain and NFTs, is not a substitute for today's Web 2.0 technology (yet). But the trend clearly changes what "literacy" means on the internet, and the new wave requires individuals to have a different mindset.

When people say Web X (X = 1.0, 2.0, 3.0, ...), I personally see X as an indicator of how the physical and digital world are strongly tied to each other; Web 1.0 represents the traditional static websites that were normally treated as a "separate world", whereas Web 2.0 & 3.0 focus more on dynamic user-generated contents that have a direct impact on our day-to-day life.

Our life is supported by the internet at a different level of technological complexity, and as X increases, the norms of real-world societies (e.g., moral and law) become more important to ensure the things comply with security, privacy, and ethics standard. That is, when we publish or consume something on the internet, it is important to have a required level of literacy not to misuse the technology and not to be manipulated by malicious actors.

Serendipity: It's Relevant AND Unexpected

Sun, 06 Mar 2022 00:00:00 +0000

As I've been discussing over the last months, I personally believe measuring non-accuracy aspects of intelligent systems is one of the most important challenges to make algorithmic recommendations ethical.

Now, let's dive deep into the most ambiguous type of non-accuracy recommender metric: Serendipity.

By definition, we can say recommendation is serendipitous if and only if it's (1) relevant to a user AND (2) unexpected for them.

First, a serendipitous item should be not yet discovered and not be expected by the user [unexpected]; secondly, the item should also be interesting, relevant and useful to the user [relevant].

Source: "Beyond Accuracy: Evaluating Recommender Systems by Coverage and Serendipity" (2010)

Hence, the practitioners need to consider what defines relevance and unexpectedness of a recommended item, on a case-by-case basis. This is a big open-ended question, and that's why the concept of serendipity is rarely discussed both in academia and industry.

Recommender Diversity is NOT Inversion of Similarity

Sun, 27 Feb 2022 00:00:00 +0000

In the modern personalization systems, diversifying what's recommended to individual users is crucial not only for maximizing customer satisfaction and business metrics but incorporating proper ethics and fairness into the applications. Here, to make constructive discussion in terms of what defines the goodness of recommendation, quantifying the concept of diversity in the form of metric is an important area of research and development.

I strongly believe the metrics discussed in academia are still far from the reality of diversity in a true sense; it is not an opposite concept of similarity, and diversity-accuracy should not be considered as a simple trade-off relationship.

How to measure diversity

To understand better about how to formulate diversity and how the research domain has evolved, I surveyed several highly cited papers, including but are not limited to:

Originally, in the early 2000s [1], people started looking diversity as a simple inversion of similarity; accurate recommendation is generated by capturing similar users/items by certain metrics $\mathrm{similarity}$, and diversity can be defined as an inversion/opposite of it, like $1 - \mathrm{similarity}$ or $\frac{1}{\mathrm{similarity}}$ (i.e., "dissimilarity").

Validate, Validate, and Validate Data. But, in terms of what?

Sun, 20 Feb 2022 00:00:00 +0000

When it comes to modern machine learning and data analytics applications, I cannot stress the importance of data validation enough. However, it's rarely discussed what defines the validity of our data.

Most importantly, accuracy, which many practitioners can easily think, is just a single aspect of the problem, and I strongly believe privacy, security, and ethics measurements must be equally treated as the accuracy metrics. Is our job done once we confirm a statistically significant increase in recall/precision and/or certain business metrics? No, absolutely not. On top of that, we (as a modern data-driven developer) must be more conscious about individual data points we are interacting with, as I discussed in Data Ethics with Lineage.

That is, the developers need to implement a way to ensure if the data is truly "good" to use. For instance, if the data contains some PIIs, your machine learning model can reasonably show better performance, but the model must not be deployed from the privacy standpoint. Or, when the data is highly skewed toward a certain population (e.g., by gender, country, religion), prediction results must be biased. These situations are carefully treated by a proper mechanism embedded in a data pipeline.

"Out of Scope"

Sun, 13 Feb 2022 00:00:00 +0000

This week, I have created a new page takuti.me/now. This will be a place where I share what I'm doing and focusing on, updated roughly once every 1-3 months. The idea is inspired by Derek Sivers's /now page movement; the movement has spread rapidly, and many people are actively sharing their immediate priorities.

Think of what you'd tell a friend you hadn't seen in a year. [...] That's what a now page is for.

I see this page is particularly useful not only for providing general updates about myself but more as a tool for maximizing my focus level and saying NO on a variety of occasions, as Derek Sivers also pointed out in the article.

I strongly believe deciding what NOT to do is equally or more important than choosing what to do. Without having solid criteria of rejection, there are infinite amounts of "opportunities" in our day-to-day life, and they simply confuse and disable us to make meaningful progress toward the ultimate goal we're originally trying to achieve. Notice that the point overlaps with what I recently described in "Definition of Done", and this is another underappreciated concept in project execution—"Out of Scope"¹.

"Definition of Done"

Fri, 04 Feb 2022 00:00:00 +0000

I strongly believe having solid "Definition of Done" is critical at a wide variety of occasions, including not only at business but for our personal life, but it's underappreciated in practice.

First, life is short, and we cannot do everything we would like. Hence, it is important for us to define a finish line by ourselves in terms of our entire life, year, month, week, and day. Otherwise, we will never get a sense of accomplishment, and we can easily miss what matters most. Although finding key factors that define an ultimate goal is not straightforward, thinking of our personal identities could be a reasonable first step to determine a high-level direction; personal identity depicts what the most important things for our life are, and such a north star metric naturally enables us to prioritize/assess the things in front of us.

Secondly, when it comes to product development, product managers should be responsible for defining a clear goal of product and leading engineers/designers with a strong ownership while sitting side by side and communicating closely with them, as well as customers. However, it rarely happens as far as I'm aware of. Design, technology, and science are essentially based on continuous effort of tiny improvements; if you gave an ambiguous goal/timeline and vague vision, they could spend an infinite amount of time.

Data Ethics with Lineage

Sun, 30 Jan 2022 00:00:00 +0000

Nowadays, there is almost no question that data is the new oil. But, do we really have a clear picture of where the oil is coming from, in which route, by whom, how, and when?

In practice, the massive continuous flows of the oil have forced us to radically automate and abstract the complex data pipelines. Eventually, we become able to "forget" about the data sources and intermediates both in a positive and negative way. Here, I strongly believe we (as a modern data-driven developer) must be more conscious about individual data points we are interacting with, and I see data lineage as a means of ethical product development in this regard.

To give an example, if you own a complex data pipeline on Apache Airflow, the tool will nicely automate your day-to-day ETL processes and hides deep contexts about the data.

On one hand, the effort unlocks developers to focus purely on an essential problem, which is commonly tied to an ultimate destination of the data flow. That is, abstraction and automation improve developer's productivity.

The Producer-Consumer Gap

Sun, 23 Jan 2022 00:00:00 +0000

I have been practicing the rule of three on a variety of occasions (e.g., business, research, blog, product narrative) for 10 years since (if I remember correctly) "The Presentation Secrets of Steve Jobs" first told me that having a clear list of 3-4 key messages has made Steve's presentation attractive.

But I frequently feel 3-4 are still a lot. Here, a main point of this article would be we need to radically distill the messages to one essential idea. Yes, this one thing.

We abstract a lot

Humans are too good at abstracting complex information and instantly processing them without the details. For instance, we can apply this mental/behavioral shortcut as a tool when it comes to habit-formation—An desired action will be automatically triggered if the action is obvious, attractive, easy, and satisfying.

Meanwhile, in the context of product development and content creation, these characteristics commonly surface as a gap between "producers" and "consumers," which can be read as any type of sender-receiver relationships such as reader-writer, artist-audience, engineer-user, and designer-client; one side cares so much about the details, whereas the receivers rarely recognize them when they interact with a deliverable.

"Diversity" Means More Than What We Typically Think

Sun, 16 Jan 2022 00:00:00 +0000

It clearly goes beyond how it’s been normally discussed in business, and that’s why my recent interest is outside of traditional accuracy/business metrics that we can “easily” optimize.

Recommendation.jl v0.4.0: Working with Missing Values, Data Typing, and Factorization Machines

Sat, 08 Jan 2022 00:00:00 +0000

This week I have released version 0.4.0 of Recommendation.jl, a library for recommender systems in Julia. As usual, you can get the package from the Julia centralized package repository and play with it as follows:

julia> using Pkg; Pkg.add("Recommendation")

It's been more than two years since the last version was released (and I presented at JuliaCon 2019). This rework is in fact part of my new year's resolution, which is about "writing more code to deliver actual values in the form of products".

Working on recommender systems in the scientific computing-focused programming language always brings a fresh perspective, and the experience enables me to rethink of how recommender interfaces should be in terms of e.g., type, hyper-parameter representation, function naming, computational efficiency; in our day-to-day work as a developer, it's easy to overlook such essential aspects when we aggressively leverage well-developed application-oriented tools such as scikit-learn and Spark MLlib.

In this release, there are three key updates I would like to highlight and want you to think of.

Ethical Product Developer

Sat, 01 Jan 2022 00:00:00 +0000

I'm consciously calling myself as a "product developer" to emphasize a desire that I am not just a software developer, data scientist, product manager, or machine learning engineer. I'm working on a product, rather than contributing to one of many random software or algorithms. I'm developing, rather than holding my own ideas, skill sets, and knowledge inside of myself.

My motivation behind learning and applying the sophisticated technologies is all about delivering an actual value to someone somewhere in the world.

More importantly, I have been seeking a way to improve myself as an "ethical product developer". To the best of my experience, there is a clear boundary between ethical and vanilla product developers¹.

For me, ethical product development requires us to have a clear answer to each of the following six questions:

Who will interact with the product?
Where is a place the product brings an impact on?
On which occasion is the product supposed to be used?
Why do I have to build this product?
What is the purpose of this product?
How does this product change the "world" both in positive and negative ways?

Who, where, which. Ethical product developers have a clear picture of end users. Selfish product developers don't care about a situation where their product is actually used by someone else somewhere outside of their control. You may read this point as "have a clear persona" in the context of design thinking, but it is important to note that the reality matters; it's not about how day of a synthetic user looks like, and an ethical developer rather tries to imagine real people in the real world and illustrate the picture as precisely as possible.

My 2021 Annual Review: A Year of Divergence (And Slowly Started Converging)

Sun, 26 Dec 2021 00:00:00 +0000

First of all, I'd like to thank everyone who has read my blog posts this year and commented on LinkedIn, Twitter, or even during a virtual/in-person chat. Your perspective mattered a lot, and I was able to effectively clarify, solidify, and sometimes refine my personal ideas as a result of these interactions. Thank you.

To make 2022 even more fruitful, let me reflect the year based on the Keep-Problem-Try retrospective framework. The format is also inspired by Annual Reviews by James Clear, the author of Atomic Habits that gave me a really nice insight recently as I summarized in "Quality Habit-Formation: It's Like Casting a Vote on My Desired Identity".

Note that I intentionally focus on my desired identifies, "product developer", "minimalist", "traveler", and "hiker & runner", and evaluate the year based on how my behaviors were aligned with them.

Keep: Things that went well & I will continue

I highly value an action that adds randomness/uncertainty and brings something new to my life. In this regard, 2021 was definitely one of the most successful years in my life.

Quality Habit-Formation: It's Like Casting a Vote on My Desired Identity

Sat, 11 Dec 2021 00:00:00 +0000

I rarely read the same book multiple times, and in 2021, Atomic Habits was one of them. (Actually, it was an Audible version narrated directly by the author (!))

Frankly speaking, the best part of the book for me is NOT about "How to create (remove) good (bad) habits", which consists of 4 principles—Making it obvious, attractive, easy, and satisfying; my experience below tells me I have no issue in forming and sustaining new habits, as long as it brings a clear benefit to my life.

Avoid alcoholic drink and caffeine;
Track what I ate;
Make audio journals;
Meanwhile, I disconnected from the internet for a whole day once a week in November.

Yes, the point is in the meaning and purpose of individual habits. In fact, I intentionally stopped (as opposed to just forgetting) some of the habits simply because I felt "I'm done" for no reasons.

Why identity matters

In Atomic Habits, I value a statement that having a strong identity is the core of habit formation most. In particular, the importance of our own identity and its relationship to habits become clear when we undergo a thinking process as follows:

How Can Recommender Systems Contribute to Mitigate Echo Chambers and Filter Bubbles?

Wed, 24 Nov 2021 00:00:00 +0000

As I summarized in "User-Centricity Matters: My Reading List from RecSys 2021", the field of recommender systems has clearly entered a new phase of research trend that focuses more on their downstream impact, and it is particularly interesting to see that RecSys 2021 had a dedicated session called Echo Chambers and Filter Bubbles with three of the following papers:

Why echo chambers and filter bubbles matter

As a result of algorithmic recommendations, information a user will interact with can be easily biased (and even manipulated by a malicious service provider). Such a controlled, enclosed flow of information is known as filter bubbles; the situation naturally decreases the diversity of data sources and eliminates a chance of knowing something new as a consumer. Ultimately, the environment keeps amplifying user's existing opinion, and this phenomenon is what we call echo chamber effect.

What Container and JIT Mean for Today's Supply Chain Disruption

Fri, 12 Nov 2021 00:00:00 +0000

If you have ever worked on supply chain in some way, it won't take too long to realize how complex and fragile the system is; numerous variables including environmental condition, labour force, local/global supply-demand seasonality are contributing to make the global supply chain possible, and none of them should be overlooked in order for the distribution network to flow seamlessly, just like human circulatory system without any blood clots.

That is, the whole system relies heavily on certainty and predictability, and it is the foundation of why your delivery packages arrive so fast after ordering, and you can purchase products made on a different continent/island at a local grocery store. I learned the fact in a painful way when I worked as a data scientist for a retail company, and the experience motivated me to dive deeper into the field later: "The Essence of Supply Chain Management".

However, the theory strongly assumes we are living in a normal "perfect" world, and it's obvious that the world has entered an unusual circumstance after the pandemic. Thus, the global supply chain is currently facing serious issues as widely reported, and the system hasn't been functioning well.

I Tried Audio Journaling for 30 Days

Sat, 06 Nov 2021 00:00:00 +0000

One of the biggest drawbacks of returning to software engineer from product manager is a significant decrease in the frequency of human interactions. I was fortunate as I became a product manager right before the pandemic and had numerous back-to-back (virtual) meetings with colleagues & customers; I believe I was able to sustain my mental health during the stay-at-home period simply because of daily communications the job forced me, even though a meeting itself did not necessarily have to be enjoyable and productive.

Importantly, the experience told me that verbalizing my own feelings, emotions, and ideas is a crucial action for me to stay motivated and live a fruitful life. That's why I keep writing blog articles on a weekly basis (either in Japanese or English) especially after the pandemic, but I still miss the abundant speaking opportunities I had had in the past. In particular, if you are single and living alone, there is literally zero chance to speak something except at work, which easily makes you depressed.

Cross-Framework Dependency Injection with spring-guice

Sat, 30 Oct 2021 00:00:00 +0000

Dependency Injection (DI) is a well-known design pattern that creates and binds dependent objects outside of a class. The technique nicely decouples dependencies from a main application class and enables developers to achieve high testability, maintainability, and extensibility. As I understand, Google Guice and Spring Framework are major DI frameworks in Java.

Unsurprisingly, working with a specific framework among others often causes a compatibility issue. Imagine you are developing an application using Guice for DI. Meanwhile, there is a third-party package that may accelerate your development effort, which actually relies on Spring unlike yours. Here, how can we apply Spring-based injection logic to Guice-based applications? An intermediate tool spring-guice could be a solution in this situation.

Dummy scenario: Machine learning application using Guice

Assume there is a Java-based machine learning framework that provides BaseModel and BaseMetric interface, and you have implemented LogisticRegression model and Recall metric on top of the framework. An ultimate goal for you is to implement the following BinaryClassification application using Guice:

My 30-Day Food Tracking Challenge from Sustainable Diet Perspective

Sat, 16 Oct 2021 00:00:00 +0000

In September, I experimentally tracked what I ate (foods, calories, and their macros) through the Fitbit app¹. The main objective was to review my eating habits and hopefully gain some weight.

In short, the practice was a great learning opportunity that allowed me to rethink how/what I should eat more consciously. Eventually, it nicely ended up with shifting to a cleaner eating habit, maintaining a consistent balance of PFC (Protein-Fat-Carb), and gaining a couple of kilograms of body weight. However, at the same time, there are also some drawbacks of the practice that I'll mention later.

Key outcomes

Here is how my typical day of eating looks like, and I have consistently kept the almost similar macro balance (P:F:C=3:2:5) for a long time.

:+1: 1. I spent less on eating

Most importantly, recording food consumption enabled me to cook and eat at home more frequently because it's simply impossible to get accurate nutritional facts otherwise, except chain brands who publish the information online. As long as you have a decent understanding of how to cook/eat well, it's certainly healthier both for your body and wallet. According to a budgeting app, I spent 35% less on the Food & Dining category in September compared to August.

User-Centricity Matters: My Reading List from RecSys 2021

Tue, 05 Oct 2021 00:00:00 +0000

Conference season is here, and RecSys is back. I've been watching the evolution of recommender systems in the last few years, along with my physical attendances at RecSys 2016 and 2018. It's great to see that the research community comes back to a physical conference unlike 2020.

After taking a quick look at the list of accepted papers, for me, one of the biggest trends in 2021 is user-centricity, which focuses on how to allow users to intervene in a recommendation process while minimizing the risk of biases and maximizing diversity & fairness of recommendations. In that sense, a list below highlights the papers that attract me the most:

Of course, this observation is "biased" by my current personal interest—Ethical challenges in recommender systems—but it's certainly an emerging area for the community as the conference has a dedicated session for "Echo Chambers and Filter Bubbles", "Users in Focus", and "Privacy, Fairness, Bias".

Understanding Big Tech's Sustainable Commitment with Word Cloud

Fri, 24 Sep 2021 00:00:00 +0000

When it comes to sustainable capitalism, the role of big companies is particularly important because they have the exceptional power of advancing sustainability in the long-run. A story that the companies like Microsoft, Amazon, Alibaba, and Google are part of the portfolio of Generation Investment Management, a sustainability-conscious investment firm, is a good example of how promising these companies are in this context¹.

Let me zoom in to Big Tech for digging deep into the reality. Today, each company has a dedicated webpage explaining their sustainability commitments:

In this post, I will examine individual pages to get a better understanding of how they look at the complex global challenges.

Sustainable Capitalism: The Importance of Long-Term Thinking and Adaptivity

Sat, 18 Sep 2021 00:00:00 +0000

There is a common myth that "sustainability" and "capitalism" are contradictory. That is, radical people tend to think either (1) global capitalism (i.e., business trends that focus heavily on innovation and efficiency) always sacrifices something important, or (2) living sustainably is unrealistic and meaningless given the modern lifestyle we've already adapted. Hence, it's easy for us to conclude that the both concepts won't be mutually achievable.

However, I personally believe it's not always true for the following reasons:

Many of the critical social/environmental problems were originally caused by technological innovations in human history (e.g., transportation, communication, mechanization), and hence making yet-another innovation driven by capital is the only effective way to truly overcome the situation. I have been a strong believer of technology for many years, and the mindset was even strengthened by Bill Gates's climate book.
As I learned in Environmental Problems Through the Lens of Business, it's certainly possible to view sustainability-related challenges as a business problem. Currently, such efforts are rarely made by limited start-up companies, but what if more investments are made in the market, and the number of players increases accordingly?

Here, I recently encountered a new concept of sustainable capitalism and had an opportunity to learn about it to deepen my thoughts further.

Voting for 2021 #BCAleTrail Best Brewery Experience Award

Mon, 06 Sep 2021 00:00:00 +0000

As a foreign worker who has been in the country only for half a year, there is nothing much to say about the ongoing Canadian federal election campaign. But don't worry, we have yet another voting opportunity: 2021 BC Ale Trail Best Brewery Experience Award.

One of the key drivers that brings me to British Columbia is its outstanding craft brewing culture. Although the activities are limited due to the COVID restrictions, I have been still able to explore many local breweries since I came to Canada in this February. So far, I'm really satisfied in this regard and have a strong impression that I made the right decision. Fortunately and unfortunately, 95% of the visits were solo though.

More importantly, I recently enjoy not only visiting breweries & grabbing a pint but also learning their story behind the scene through BC Ale Trail (though I'm not a fan of their mobile app's user experience), Cascadian Beer Podcast, and Pacific Beer Chat. Here, the Best Brewery Experience Award seems to be a great opportunity for me to look back on my journey on the BC Ale Trails and plan for the next adventure to dive deeper into them.

Save HTML

Sat, 04 Sep 2021 00:00:00 +0000

After playing with interactive data visualization using D3.js and React, I wanted to have a "Download" function so that I can easily save the visualization as an image rather than taking a screenshot of a browser window:

In short, it can be achieved by writing a JavaScript snippet that does the following job:

<svg> to Blob
Blob to Image
Image to <canvas>

Eventually, HTML Canvas enables you to save the content in the form of image file.

For the future reference, let me walk-through the process line-by-line since coming up with a complete snippet wasn't straightforward; there were some "tricky" parts that aren't fully covered by a piece of online resources e.g.:

Assume we have an SVG drawing index.html and its stylesheet e.g., style.css:

<svg xmlns="http://www.w3.org/2000/svg">
  <rect width=64 height=64 />
</svg>

svg rect {
  fill: blue;
}

SVG to Blob

The first and most confusing step is to convert SVG into a Blob object represented by XML:

Next "Dot" in Journey: Curiosity-Driven Job Change in Canada (Aug 2021)

Sat, 28 Aug 2021 00:00:00 +0000

After five years of dramatic days at Treasure Data (TD), I decided to leave the company and join Amazon's Personalization team in Vancouver as a software engineer. When I first entered TD as an intern back in 2016, I didn't expect such a long, complex, and fruitful journey. I was influenced by the great corporate culture & colleagues literally on a daily basis, and I'm sure the 5-year experience will remain as a core of my life for a long time.

Importantly, while I changed my role from Software Engineer to Product Manager 1.5 year ago, now I'm back as an engineer for the new position in the field of recommender systems, where I have worked for a few years in the past (e.g., Google Scholar, CV). What I really like about personalization systems is how theoretical aspects¹, engineering problems², UI/UX design concerns³, business impacts⁴, and ethical challenges⁵ are strongly tied to each other. Thus, I'm curious about working in the domain at Earth's most customer-centric company.

Why Your Job Title Matters

Sat, 07 Aug 2021 00:00:00 +0000

Recently, I happened to be "categorized" by my past and current job titles a lot, during the immigration and job interview process¹. This experience makes me realized that adapting to a certain way of classification defined by someone else was a little discomfort.

Here, I strongly believe that entitling someone (or something) is a highly important and sensitive problem for the following reasons. Note that, although I'm writing about job titles, I won't discuss anything about seniority (e.g., Senior Software Engineer vs. Principal Software Engineer) in this post.

Our stereotype

Most importantly, people tend to read too much from a title itself based on their stereotype. Consequently, without having enough contextual understanding, showing a title possibly gives a wrong impression about you to strangers.

To give an example, if I introduce myself with a title as "I am a software engineer", what does that mean? It should indicate nothing other than a single fact that this person is "engineering software". However, in practice, you can easily imagine something else like This person can do programming, have a computer science degree, type a keyboard faster, ... no matter what the reality is.

Reviewing Ethical Challenges in Recommender Systems

Thu, 15 Jul 2021 00:00:00 +0000

As I captured in Understanding Research Trends in Recommender Systems from Word Cloud, many real-life applications are nowadays applying recommendation techniques to a wide variety of datasets and scenarios.

However, their objective focuses too much on the accuracy of recommendations. That is, the major goal of many recommender systems is to accurately capture user preference and maximize the chance of positive user-item interactions, and hence consideration about surrounding non-performance-related problems is lacking. In that sense, I have to hesitantly say that my past studies are not an exception; over the past few years, I cared only about how to improve the accuracy of recommendations in a scalable manner.

By contrast, a survey paper "Recommender systems and their ethical challenges" shows an important concept of multi-stakeholder recommender systems and argues how today's recommender research unintentionally helps to pose ethical challenges to the world. I believe having such an ethical perspective is one of the most important responsibilities for the developers of modern intelligent systems. Notice that, since we are engaging with the advanced systems literally on a second-by-second basis in our daily life e.g., on a mobile phone, e-commerce, social networks, and maps, it's critical for all of us to rethink the definition of good recommender systems.

Understanding Array Bisection Algorithm

Sat, 05 Jun 2021 00:00:00 +0000

Given a sorted array arr, Array Bisection Algorithm (a.k.a. Bisection Method, Binary Search Method) enables us to find an insertion point i for a new element val such that arr[i-1] < val <= arr[i] (or, arr[i] < val <= arr[i+1]).

Problem

Consider we want to insert a number 10 to a sorted array [2, 4, 8, 16, 32]. Here, an insertion point should be i=3 as arr[2] < 10 <= arr[3].

A naive method is sequentially walking through the elements until we hit the condition:

def search(arr, val):
    """
    >>> search([2, 4, 8, 16, 32], 10)
    3
    """
    if val < arr[0]:
        return 0

    for i in range(1, len(arr)):
        if arr[i-1] < val and val <= arr[i]:
            return i

    return len(arr)

The time complexity of this approach is $O(N)$, and larger arrays take more time to complete the operation.

Intuition

As an optimized way to solve the problem, binary search finds out an insertion point in $O(\log N)$ time complexity.

The basic idea of the method is to repeatedly split arr into two chunks, first-half arr[:mid] and last-half arr[mid+1:] of the elements, until a dividing point mid reaches the target value val.

Hi Product Managers, Are You Creating Products That You Love?

Wed, 26 May 2021 00:00:00 +0000

Otherwise, I have a feeling that the job of product management is likely to be a bullshit job.

Meaning, no matter how experienced/senior you are and how much salary a company pays to you, the job is easily replaceable and even meaningless unless there is a strong reason why "you" have to be a product manager for this particular product.

In that sense, I believe having a personal connection between you and the product itself is equally (or even more) important to any other soft skills e.g., communication and project execution.

What I learned from my first PM experience

It's been more than a year since I changed my role from software engineer to product manager (PM), but, to be honest, I still cannot envision my career growth as a PM; it depends on a company, product, industry, and people if I choose a PM role in my next career, whereas I can clearly imagine an infinite career path as a software engineer.

Unusual Drinking & Eating Habits: Non-Alcohol, Decaf, Flexitarian

Fri, 21 May 2021 00:00:00 +0000

It's been almost 3 months after moving to Canada. Overall, Vancouver is a too good place to live because of its coffee, craft beer, and food culture. There are so many breweries, cafes, and restaurants we can enjoy drinking and eating, and my alcohol and caffeine intake have clearly been increased since I came to the country.

That said, in the first two months, I have visited almost all of the "want to go" places where I can access by walk or public transport. Hence, to make a room for exploring something different, I experimentally changed my drinking and eating habit as follows.

Stop drinking alcohol
Drink decaf coffee only
Practice flexitarian diet and make a meat free day

Let's see what the new habits brought to me.

Stop drinking alcohol

Before — Regularly drink beers (1) as a hobby and (2) to refresh. About three days a week, mostly on weekends.
After — Explore different types of non-alcoholic beer and Kombucha.
Findings
- Similar to what I found when I Stopped Drinking Alone for 4 Months
  - Eating becomes more enjoyable.
  - No visible impact on productivity and health condition.
- Canadian non-alcoholic beers taste way better than the one in my old memory.
- Kombucha addiction...
  - "Kombucha on tap" at Green Leaf Brewing or some vegan cafes makes me feel like drinking a sour beer.

As I realized there are enough alternative options to locally enjoy refreshing drinks, I probably keep the non-alcohol habit as much as possible, at least when I'm alone, unless a new brewery opens in an accessible area.

A Journey of Sustainable Development #SDGMOOC

Fri, 14 May 2021 00:00:00 +0000

Driving business towards the Sustainable Development Goals offered by Erasmus University Rotterdam was one of the best online courses I've ever taken. By showing real-world business cases, the well-structured course gave me a clear understanding of individual Sustainable Development Goals (SDGs) following the "wedding cake" model:

* Credit: Azote Images for Stockholm Resilience Centre, Stockholm University

The point is SDGs are mutually dependent; our life (society) relies fully on the natural resources (biosphere), and the economy exists on the top of human life. Therefore, there is a wide variety of opportunities for each of us to contribute to SDGs both in personal and business.

Taking responsibility in an infinite journey

Key takeaways can be explained by the quotes captured from the course videos.

First of all, the issues in ecological, social, and economical sustainability are made up of many different factors, and there is no clear "goal" that we can stop maintaining sustainability.

Sustainability is a journey and not a destination.

—Bob Collymore, A former CEO of Safaricom

Actionable Insights from Supply Chain Analytics

Fri, 07 May 2021 00:00:00 +0000

After deepening my knowledge about Supply Chain Management, I became particularly curious about how data helps supply chain management. Here, another course Supply Chain Analytics illustrated a realistic example of VASTA, one of the largest mobile network careers in the US (Verizon, AT&T, Sprint, T-Mobile), which made a big change in their distribution strategy as a consequence of supply chain analytics.

A list below is a step-by-step guide to using data for supply chain optimization:

Opportunity identification
Intuition & insight development
Data analytics to confirm the intuition
Implementation without sacrificing customer satisfaction

Notice that actual analytics work is a limited part of the entire process. As many best practices for data science explain, understanding the domain well, making a better hypothesis, and planning a direction that makes sense to you are the crucial steps to undergo. Plus, the final implementation work happens in the real world (not on your Jupyter notebook!), and hence we shouldn't forget to be considerate of what the data doesn't explicitly tell.

Environmental Problems Through the Lens of Business

Fri, 30 Apr 2021 00:00:00 +0000

One of the key insights I gained from Bill Gates's "How to Avoid a Climate Disaster" is that the next economic leader in the world is an innovator in the global environmental management field.

But, is it truly possible to balance technological innovations and business outcomes (i.e., profit)? If so, what kind of technologically and economically solvable problems we have today?

To answer the questions, I dove into the topic through Renewable Energy and Green Building Entrepreneurship and Global Environmental Management on Coursera and studied some of the real-world examples. These courses told me why environmental problems matter for all of us, how we can readily make a change, and how the effort leads to business opportunities.

What's a green building?

Green building is a holistic design of an entire building that achieves a high efficiency of (net positive) energy use, zero "red-listed" chemical use, and better city planning and social equity. For example, reusing materials, use of natural lights and energy-conscious HVAC systems make up a green building.

My First Impression After Reading Bill Gates's "How to Avoid a Climate Disaster"

Sun, 25 Apr 2021 00:00:00 +0000

I have been following Bill Gates's activities against environmental problems since I watched Inside Bill's Brain on Netflix last year, and I just finished reading his latest book, How to Avoid a Climate Disaster.

For those who are interested in the climate crisis but have no idea of what the current state is, just like me, the book can be a deeply satisfying beginner's guide to the global problem. Although I need to review the informative content again to flesh out my thoughts, I could share some of the key takeaways that lead us to immediate actions TODAY:

Growing sensitivity and intuition about numbers (facts; data) is important, to begin with.
We can be optimistic about technology, but the real challenge is in its management strategy relying on the government's policies and regulations.
The next economic leader in the world is an innovator in the global environmental management field.
Climate crisis is not somebody else's problem. It is our problem, and we can make a change.

First of all, Bill refers to many different numbers throughout the book, and having a better sense of these numbers and being able to quickly capture a big picture of individual problems would be an important first step to dig deep into a specific topic. For instance, a nuclear power source generates 500 to 1,000 watts per square meter, but...how big or small is it? The book (and this article) shows an intuition about the scale as follows:

Learning Cryptocurrency with Code

Mon, 19 Apr 2021 00:00:00 +0000

In "What Blockchain Brings to Real-World Applications", I learned blockchain basics ranging from cryptography and hashing to smart contracts and decentralized applications using the Solidity programming language. For me, one of the most important findings was the diversity of blockchain applications; cryptocurrency like Bitcoin is not the only application blockchain makes possible, and the technology resolves a wide variety of real-life problems in a decentralized manner as long as:

trust among unknown peers in a network matters;
the situation requires validation, verification, recording of time-stamped immutable conditions.

On the other hand, there is another online course "Bitcoin and Cryptocurrency Technologies" offered by Princeton University, which focuses purely on blockchain technologies for money transactions. The content is fully based on their textbook, and the course gives us highly detailed (mostly theoretical) explanations of how & why Bitcoin works.

The advanced course has covered a couple of notable topics as follows, and the coding exercises helped me to have a clear picture of how (1) transaction is verified, (2) consensus is made, and (3) block (i.e., set of transactions) is created & added to a chain.

What Blockchain Brings to Real-World Applications

Sun, 11 Apr 2021 00:00:00 +0000

I have completed Blockchain Specialization offered by University at Buffalo on Coursera.

So, what is blockchain?

It's a set of techniques that makes secure P2P transactions possible among distributed nodes in a decentralized system. By leveraging the methods for validation, verification, consensus, and immutable recording, blockchain allows us to establish trust with unknown peers.

That is, if trust among unknown peers in a network matters and the situation requires validation, verification, recording of time-stamped immutable conditions, blockchain can be a deeply satisfying way to build an appropriate solution to the problem.

Motivation

This specialization became my first hands-on blockchain experience, and I was particularly curious about how the concept is applied to real-world problems.

To give an example, Supply Chain Management Specialization told me that a core of real-world supply chain is all about a holistic optimization process from sourcing and planning to operations and logistics. It means that establishing trustable, efficient mutual relationships among individual roles (e.g., procurement department, supplier, truck driver) is a fundamental requirement for the whole system. Here, connecting people and relaying assets can respectively be considered as nodes and transactions in the blockchain world, and blockchain techniques help to make the process more transparent and reliable for everyone ranging from a raw material supplier to a consumer.

The Essence of Supply Chain Management

Sun, 04 Apr 2021 00:00:00 +0000

On Coursera, I have completed Supply Chain Management Specialization offered by Rutgers University, which consists of four different topics in an end-to-end supply chain management process:

Logistics
Operations
Planning
Sourcing

The series of courses gave me a clear picture of how real-world companies build and operate supply chains, and what kind of practical problems exist throughout the process.

For me, a key takeaway from the specialization can be three-fold:

There is always a trade-off between service level, risk, and cost.
To make the right decision, having a solid corporate-wide objective, mission, and strategy is important.
Starting from the simplest possible approach and making continuous improvement lead an organization to success.

Sounds like general statements to run a business? I know, but this is exactly how the real-world supply chain runs.

As a data scientist, I have worked for a couple of companies that are struggling with supply chain optimization in the past. Of course, I could tackle their problems without having domain-specific knowledge, by simply applying general data science & analytics practices from EDA (Exploratory Data Analysis) to statistical modeling. However, if I knew the basic concepts and best practices taught by the courses, I could do a better job and more confidently present the insights to convince stakeholders.

Moving to Canada: Adding Randomness to My Life

Sun, 21 Mar 2021 00:00:00 +0000

On Feb 26, 2021, I have officially moved to Metro Vancouver in British Columbia, Canada from Japan under continuous employment with Treasure Data, where I have been working for 4+ years.

Why?

Just as an experience, seriously.

Of course, there are some tactical reasons as follows:

I have become a Product Manager having data science expertise. See "Why a Data Science Engineer Becomes a Product Manager" for the background.
Over the past few years, I have been interacted with many valuable customers across the world not only as an engineer but occasionally acting as a consultant, pre-sales person, and product representative.
As the company's business grows, the Canadian entity of Treasure Data currently plays a key role as the center of their R&D activities. Here, my experiences above hopefully help to accelerate the growth by bringing customer's points of view to the local team.

By the way, Treasure Data Canada is hiring :)

That said, my personal motivation is simply experiencing life in a foreign country, and I've been seeking the opportunity over the past few years.

How Much CO2 Emissions Have Your Flights Made?

Sat, 13 Mar 2021 00:00:00 +0000

Before reading further, play with an interactive data visualization app that I implemented with D3.js & React: Flight Emissions Calculator

It's been a while since the COVID-19 pandemic arose, and one of the biggest changes in my life was a significant reduction of air travel opportunities both for business and private. This fact is very unfortunate for me because traveling was my greatest joy in life.

https://twitter.com/takuti/status/1207165848135168002

However, at the same time, there is a positive side in terms of the environment; it's obvious that air travel is environmentally unfriendly due to its significant amount of CO2 emissions, and hence fewer flights certainly reduce our overall carbon footprint.

Visualizing flight emissions

To realize the fact, let's see how much CO2 emissions my flights in 2019 have made:

In this figure, colored countries represent that the CO2 footprint of my flights in 2019, which was about 24.239 tonnes, exceeded their per-capita yearly emissions. That is, I flew too much during the year and made more CO2 emissions than the personal averages of almost all countries in the world. For instance, a single person in Japan produced 8.7 tonnes of CO2 in 2019 on average, and my flight emission was greater than the number. It's shameful, and I shouldn't be proud of myself for being a frequent flyer.

Moving Julia Project from Travis CI to GitHub Actions #JuliaLang

Sat, 06 Mar 2021 00:00:00 +0000

Thanks to a pull-request "Install TagBot as a GitHub Action", which was created by a bot, I realized now is the time to move one of my Julia projects, Recommendation.jl, from Travis CI to GitHub Actions.

Since I haven't had enough time to work on the project for a year, migration was not straightforward as I expected. Hence, this post summarizes what I've undergone to get there. As always, I referred to some of the most actively/recently maintained official Julia packages (e.g., Statistics.jl, Tokenize.jl) to see what is the most 'modern' way to organize a Julia project.

Original `.travis.yml`

I've written the following Travis CI configuration more than a year ago:

language: julia 

os: 
  - linux 
  - osx 

julia:  
  - 0.7 
  - 1.0 
  - nightly 

matrix: 
  allow_failures: 
  - julia: nightly  

notifications:  
  email: false  

after_success:  
  - julia -e 'using Pkg; Pkg.add("Coverage"); using Coverage; Coveralls.submit(process_folder())' 

jobs: 
  include:  
    - stage: "Documentation"  
      julia: 1.0  
      os: linux 
      script: 
        - julia --project=docs/ -e 'using Pkg; Pkg.develop(PackageSpec(path=pwd()));  
                                               Pkg.build("Recommendation"); 
                                               Pkg.instantiate()' 
        - julia --project=docs/ docs/make.jl  
      after_success: skip

The workflow breaks down into three pieces:

Unit testing over multiple Julia versions
Profiling test coverage
Building and publishing a documentation page to GitHub Pages

Running a Workflow over multiple Julia versions

First and foremost, what I like about GitHub Actions is the easiness of using a build matrix.

Practicing D3 Interactive Data Visualization with Fitbit Activity/Sleep Log

Wed, 03 Feb 2021 00:00:00 +0000

As I introduced in Datavis 2020: A Free Online Course About D3.js & React, the online course told me the basics of how to effectively and efficiently create interactive data visualization using D3 and React.

It's time to practice, and I have built Fitbit Activity/Sleep Explorer putting everything that I learned from the course together, including but not limited to a bar chart (binning and brushing), scatter plot (hovering on color legend, selecting axes from the dropdown), memoization technique.

Starting from problem statement & sketch

For all of the real-world data problems, we must set a clear goal before working on the data in front of you. Otherwise, you'll just waste your time, and nobody loves your deliverable due to an incorrect choice of dataset and/or inappropriate way of representation.

Here, I've been continuously curious about the relationship between activity and sleep since I purchased my Fitbit Charge 3 and started tracking the records. Therefore, one of the questions that I tried to get an answer from visualization was: Does being active lead to better sleep?

Datavis 2020: A Free Online Course About D3.js & React

Sat, 23 Jan 2021 00:00:00 +0000

I have recently studied data visualization with D3.js and React from Datavis 2020 by Curran Kelleher.

Here is a demo page I deployed the outcomes: takuti-sandbox.github.io/datavis-2020 [repo]

What Datavis 2020 introduces

Overall, the course was just AMAZING and does teach us a wide variety of essential knowledge, including not only basic usage of D3.js but advanced performance optimization techniques:

Key concepts in data visualization e.g., types of data and chart, starting from a hand-writing sketch and communicating with stakeholders
Git basics
Tracking license and source of a public dataset
React and ES6 basics
Creating modules
Performance optimization by memoization

That is, I believe the video series is also beneficial as an 101 course for programming and software engineering.

As Curran referred in a video, Datavis 2020 follows the following principle:

Make it work, make it right, make it fast. — Kent Beck

One thing I like about the course is how the contents are well-structured from the basics to advanced topics in this particular order.

Are You Still Writing a Resume in Word/LaTeX?

Sat, 28 Nov 2020 00:00:00 +0000

Over the past few years, I have been using LaTeX for writing a resume.

While it looks so professional especially in the academic world, there is a space for improvement in terms of maintainability, flexibility, and customizability.

Thus, I have started writing a resume in HTML and publishing it in the form of both PDF file and static website:

takuti.me/cv
GitHub repository

Notice that the implementation is fully based on lucas-clemente/cv. The template builds a static resume website by using Broccoli.js, and a PDF version is dynamically rendered via Screen Capture with PhantomJS. In fact, the code is slightly out of date, but it's fine and I can further tune and tweak as much as I want because it's just a simple HTML and JavaScript stuff.

Pros

Most importantly, I don't necessarily have to install TeX on my laptop anymore. After I graduated from the master's program and joined a company as an engineer, I have had a very limited opportunity to write an academic paper in LaTeX (only twice in the last 3-4 years¹²). That is, resume writing was the only reason why I install the TeX environment every time I set up a laptop. It was so frustrating for me as it keeps causing problems and requires spending quite a bit of time on resolving them. HTML and JavaScript, on the other hand, are straightforward that make the situation a lot easier and simpler.

What I Think About When I Talk About ML Product

Sat, 29 Aug 2020 00:00:00 +0000

Everyone wants to leverage machine learning (ML) in their daily work, and I'm sure they will be excited about your awesome ML product whatever the detail is. But, what do we, as a product developer, really have to think about when we create ML products?

Based on my experiences and what I learned from the following insightful articles, let me summarize five key questions we should ask ourselves.

1. Do we have a clear mapping between business and technical problems?

Regardless of whether it's ML-based or not, our product should translate high-level business requirements into specific technical problems.

Since "AI/ML" sounds exceptionally attractive but cannot solve all the problems in the world, making sure the connection is particularly important.

For example, if our business requirement is to improve product margin, a technical problem our product handles is solving an optimization problem to find an optional product price.

Language as a Design Tool

Sun, 16 Aug 2020 00:00:00 +0000

As I wrote in Don't "Guess" How People in Other Roles Work, I have been studying something new in my spare time, and the two online courses I have recently accomplished, Visual Elements of User Interface Design and UX Design Fundamentals, told me an important fact that language is a very powerful tool for design.

Non-visual description of idea accelerates design decisions

In Visual Elements of User Interface Design, we studied how to define a mood of an app, choose a visual direction of its interface, and make some design decisions. Importantly, even though the ultimate goal is to design visual elements as the course name suggests, our first week purely focused on non-visual descriptions of an idea.

We designed a smartphone app throughout the 4-week assignments, and the very first task was to provide a short description of an idea that answers to the following three questions: What is it, Who is it for, and Where does it live?

Based on the description, we then determine high-level concepts of the app; I designed a heartbeat monitor app for software engineers, and the basic concepts were:

Don't "Guess" How People in Other Roles Work

Sun, 31 May 2020 00:00:00 +0000

Spend a bit of your time to "learn" what they do, and talk to them, instead.

"Two courses every product manager should take"

In an audiobook "Inspired: How to Create Tech Products Customers Love, Second Edition¹" I've recently listened, the author stated that there are two courses every product manager should take:

Introduction to Programming
Introduction to (Corporate) Finance

It doesn't mean product managers should be capable as a professional engineer or financial specialist.

The point is that, as long as you're working in a cross-functional team, there is a clear need for effectively and efficiently collaborating with the others, and hence knowing their way of work & its difficulty makes the teamwork way more successful than unknowing.

Problem of guessing

For instance, the following scenarios show our work is clearly based on mutual collaboration among the team members working in different areas:

Product managers need to assess value, risk, and business and technical feasibility for prioritization.
Engineers want to know the accurate timeline and requirements for an application they're developing.
Designers have to deeply understand the users to deliver the right experience using the right technology.
Finance officers (and/or executives) must be clear on the risks and values tied to the product features, in consideration of corporate-wide business objectives.

Sales, marketing, legal, customer success — There are numerous variations of inter-role dependency in practice.

Completed Functional Programming in Scala Specialization on Coursera, Finally

Tue, 26 May 2020 00:00:00 +0000

I have earned my certificate of Functional Programming in Scala Specialization completion.

To be honest, I finished 4 out of the 5 specialization courses 3 years ago, when these were still independent free courses, as you can see my notes in the following articles:

Recently, I noticed that the courses are now packaged as Specialization on the platform, and I can be certified by Coursera & EPFL if I complete a remaining Capstone Project. Well, Coursera offers a 7-day full access free trial for the Specialization, and we do have enough time to complete the coursework thanks to COVID-19. Why don't you spend the whole weekend to grab the low-hanging fruit?

I did that last weekend.

It was a good refresher

A goal of the Capstone Project is to implement a practical Scala application that:

Loads a large volume of historical temperature data,
Applies some numerical computation techniques for analyzing geospatial data & interpolating values of temperature,
Visualizes the historic global temperature maps in an interactive web application.

Sounds fun, doesn't it?

Issue-Driven Makes You Professional

Fri, 01 May 2020 00:00:00 +0000

What does define "professional" mean?

Salary
Years of experience
Job grade
Skill
Number of publications
...

All of the above factors could be true. But, more precisely, I believe the quality of what you deliver determines the degree of our professionalism. On that point, one question comes to my mind — Given the 24 hours and personal constraints, how can I maximize the quality of deliverables? Note that the constraints are derived from a variety of personal conditions associated with money, health, family, education, etc.

Most importantly, the amount of time spent on a problem does NOT always increase the quality of outcomes. I recently watched "Inside Bill's Brain" on Netflix, and Bill repeatedly says Work harder in the video series; he emphasizes the importance of spending more time on a problem, and the approach works surprisingly well because of Bill's exceptionally deep concentration and complex problem-solving skills. However, we're not Bill, and the approach probably doesn't work for most of us due to so many distractions in reality and our limited ability to focus.

How I'm Working From Home

Sat, 18 Apr 2020 00:00:00 +0000

We have spent more than a month since working-from-home (WFH) accidentally becomes a global trend. Although I'm still actively adapting my life to the situation day by day, I already got a few solid takeaways from the experience as follows:

Essential things to buy
Key habit to stay mentally and physically healthy
How to balance work and home life

When I was a master's student, I used to go to a university every day for research activities even if I don't have any classes or meetings on that day. The situation doesn't change until today; in fact, my company is flexible whether I work at an office or from home, but I was commuting to an office at almost the same time every day. It means that I was completely unprepared for entering WFH this February. Moreover, since I was traveling a lot for the past two years and be outside of the country for over half a year, my home had very limited facilities to do something productive.

What Makes a Good Dashboard: The Rise of Augmented Analytics

Sat, 11 Apr 2020 00:00:00 +0000

A good dashboard is all we need — As a person working in the field of data-driven business, I strongly agree with the statement written in "One Thing to Maybe Build Now: A Great Dashboard".

How to build a good dashboard

Thanks to the recent advance of data analytics, building an intelligent "shiny" application becomes easier. In the business context, however, the end-users do not always have sophisticated data science background to understand what happens behind the scene and perform advanced analytics. Thus, as the article mentioned, a good dashboard is defined by how a UI helps (and even automates) an entire process of visualization:

Requires zero work from the dashboard consumer. The data is automatically generated by the app.

That would be awesome if we could build a meaningful dashboard consumed by the beginners with zero configuration. But, how?

Here, recent trends in Augmented Analytics can be a solution. On the Augmented Analytics applications, data is explained for / searched by citizen data scientists, who might be capable to perform basic analytics but do not have advanced skills, unlike professional data scientists.

Life with (Partially) Broken iPhone

Sat, 04 Apr 2020 00:00:00 +0000

Recently, I dropped off my iPhone from my hand, and the screen has partially been broken. It was definitely sad news, and I got really panicked at that moment. On the other hand, it's not a first-time experience for me, and having a broken iPhone is always a great opportunity to rethink how I interact with the magical devices. I strongly believe our life relies too much on the devices, and most of your "essential" applications are not actually essential.

Recognizing unnecessary apps

Most importantly, when I become unable to use a smartphone, I immediately realize how my life does depend on the device (but doesn't, in fact) and how much time I'm spending on the internet.

This is how my iPhone's home screen looks like (yes, mine has only a single screen):

This time, the upper half of the touch screen has completely died. It means that I was NOT able to...

read and send a message on the communication tools, including Slack, WhatsApp, and Facebook Messenger,
check my Marriott Bonvoy or Hilton membership status and booking information, as well as some of my key financial status associated with the bank accounts,
login to web services via 2FA,
read online articles or Kindle books,
listen to music,
check the latest weather, etc.

Plus, even though opening apps in the lower half part (e.g., Google Chrome, Maps, and Gmail) was fine, using them as usual was very difficult as you can easily imagine.

The Hardest Part of 2020's Strategic Ideation

Sun, 29 Mar 2020 00:00:00 +0000

In How to Produce Ideas, I emphasized the importance of gathering as much information as possible from a wide variety of sources. On that point, the current chaotic situation in the world suddenly makes it easier, since we have less social interactions and more time to concentrate on private activities such as reading books, learning something new, and deep-diving into essential things in your life. For me personally, this is definitely a positive side of the crisis, and I'm taking full advantage of the quiet period by proactively spending my time on ideation both for private and work.

However, I just realized that, once we got a lot of valuable inputs from both online and offline sources, the next step "Staying away from the problem, like fermentation" would be far more challenging and important to solidify the thoughts for the following reasons.

Too much concentration makes me narrow-minded

First of all, if we really focus on a single topic that motivates you a lot, we won't have a chance to temporarily stay away from the problem and can easily lose an objective view as a consequence of over-concentration.

Why a Data Science Engineer Becomes a Product Manager

Fri, 07 Feb 2020 00:00:00 +0000

As an engineer, I've been working on machine learning (ML) and data science at Arm Treasure Data over the past three years. You can find more about me in the portfolio page. Basically, my job responsibility, which overlaps with my personal interests a lot, has been somewhere between engineering and customer-facing work for bringing advanced technologies such as ML and IoT to an enterprise-grade big data analytics platform.

Based on the experiences that I had during the three years, I recently decided to changed my role from Engineer to Product Manager (PM) within the same organization. This post explains the reason why I made the decision. In my opinion, the PM is a unique option for those who are looking for an opportunity of delivering real-world ML solutions.

TL;DR

I've been continuously looking for a way to bridge a gap between scientific theory and real-world practice (e.g., ML algorithm vs. its application).
My experience in the industry told me that considering user experience is crucial to successfully productize complex technologies.
To formally practice user experience-focused product development in the ML and data science domain, I have decided to switch my role from Data Science Engineer to PM.
Product management is one possible, lesser-known way to achieve meaningful outcomes by fully leveraging your ML and data science expertise.

My backbone: Bridging a gap between theory and practice

Most importantly, I have had a strong motivation on filling a gap between scientific theory and real-world practice for a long time, and I've always chosen a position where I can work in the middle of scientifically and technologically stimulating techniques and advanced, innovative applications.

Design Thinking = Capturing the World Like a Designer

Sun, 26 Jan 2020 00:00:00 +0000

Everyone loves design thinking, a systematic way of problem-solving, and, in practice, the approach suggests non-designers to undergo the following steps to efficiently and effectively build valuable solutions:

Empathize with users.
Define problem to be resolved.
Ideate solution (by brainstorming).
Prototype the ideas to give feedback.
Test prototyped idea with users.

Real-world problem solving is more complicated

Here is a question — Can we really capture and resolve essential problems by this simple step-by-step procedure? I can easily imagine the answer is NO.

To give an example, a talk "Design Thinking Is Bullsh*t" argued that our world is not that simple as it can be summarized by the pieces of Post-it paper. According to the speaker, a designer's way of problem-solving is more like a back-and-forth process of improvements based on iterations of creating, validating, and criticizing. Thus, the widely known high-level definition of design thinking approach is somewhat different from the reality, and, in the context of product development, it is important for the developers to carefully understand what design thinking actually means and how we can build successful products on top of that.

The Rise of Customer-Centric Retailing @ NRF Retail's Big Show #NRF2020

Mon, 20 Jan 2020 00:00:00 +0000

My 2020 has been kicked off with NRF 2020, one of the world's largest retail conferences. In the 3-day international event held in New York, USA, a lot of exhibitors and speakers ranging from key stakeholders in the famous brands to engineers building state-of-the-art AI/IoT systems got together and shared their challenges and solutions.

When I entered the venue, I was just surprised at how big NRF is; there were three floors dedicated to the Expo, and everywhere was so crowded!

Here are key takeaways I got from the event:

The industrial trend is clearly in "customer" (e.g., customer data, customer experience, customer insights).
Following the end of retail apocalypse, brands are now focusing on seamless digital-physical customer experience, especially for the millennials who require retailers to conduct hyper-personalization and effective storytelling.
At Expo, many point solution vendors were helping the brand's specific problems (e.g., in-store face recognition, demand forecasting, supply chain optimization, customer data management). On the other hand, I saw only a few platform solutions which allow us to integrate the physical and digital world in one place.

Keynote sessions

The very first session I've attended was a keynote talk by Microsoft CEO, Satya Nadella. He nicely reviewed emerging topics in the retail industry such as personalization, intelligent supply chain, and retailtainment. To effectively and efficiently overcome the real-world challenges including everywhere commerce and data silo, he introduced various Microsoft solutions and their client's use cases:

How to Produce Ideas

Sun, 05 Jan 2020 00:00:00 +0000

Producing ideas is an important process of creating something new (e.g., business, product, software, article), but its methodology is not trivial. On that point, I strongly believe that collecting more information and experiences is a key as follows.

Seeds of Ideas

First of all, I have just read James Young's book "A Technique for Producing Ideas" which introduces five essential steps to strategically produce ideas, and he stated that gathering raw material comes first:

Gathering raw material
Finding out a relationship between them
Staying away from the problem, like fermentation
An idea (unexpectedly) comes up in your mind
Implementing the idea, and evaluating its effectiveness

Here, raw material includes a wide variety of experiences in our life, tons of news articles and knowledge about art and history we can access both online and offline, and specific knowledge corresponding to your problem and customers. All of these inputs could eventually be the seeds of new ideas.

Since a new idea is normally a combination of existing pieces of experiences, knowledge, products, and someone else's ideas, proactively gathering the seeds is highly valuable to make great ideas.

What I've Seen at IoT Solutions World Congress 2019

Sun, 17 Nov 2019 00:00:00 +0000

October 2019 was a big month! After the two exciting conference opportunities, ApacheCon Europe and Data Leaders Summit, in Berlin, I flew to Barcelona, Spain and joined IoT Solutions World Congress 2019 (IoTSWC).

https://twitter.com/takuti/status/1185419609333272576

As a data science engineer who has developed digital data-centric products, I learned a lot from the IoT conference to see a big picture of the industry mixing the physical and digital world.

TL;DR

For me, there were three particularly important findings:

Everyone is suffering from the gap between PoC and its productization.
Machine Learning (ML) is everywhere, and it's not THAT special anymore.
Blockchain is becoming a key piece of modern IoT systems.

Biggest Challenge: PoC to Productization

PoC-ish initial implementation was done in a couple of months, but a project team is now spending 2-3 years to make the PoC implementation production-grade.

Sounds like such a situation is very likely to happen in real-world IoT projects.

Thus, some conceptual talks and panel sessions equally mentioned how to overcome the gap by building a solid team and effectively leveraging the resources & surrounding technologies. Across the entire conference including Expo, the audiences were actively looking for third-party solutions and partners to accelerate time to value.

Rethinking the Role of Data Leaders @ Data Leaders Summit Europe 2019

Sun, 10 Nov 2019 00:00:00 +0000

After my speaking session at ApacheCon Europe in Berlin, luckily I've got a chance to attend Day 2 sessions of Data Leaders Summit Europe:

https://twitter.com/takuti/status/1187262382944538624

As the name suggests, speakers and audiences of the summit are the leaders of data division within a company such as Head of Data and CDO, who are in a decision-making role, and it was a great opportunity to identify the difference between data leaders and data "players" (e.g., data scientist, analyst, ML engineer), and rethink their responsibilities.

In the sessions, there were two particularly important topics:

How to build data organization(s) within a company, in consideration of diversity.
How to make real-world data project successful.

Good Data Scientist != Good Data Leader

First of all, a key responsibility of data leaders is to translate business needs into data requirements, and successful data organization and project must be led by those who are capable to accomplish it.

On that point, it is important that good data scientists cannot always be good data leaders; a data scientist is a person who is working on data itself to make the outcomes, whereas a leader is in charge of the more strategic decision-making process before aggregating data and playing with it.

ApacheCon 2019 North America #ACNA19 & Europe #ACEU19

Sat, 26 Oct 2019 00:00:00 +0000

As a committer/PPMC of Apache Hivemall (incubating), I have attended and presented at ApacheCon North America and ApacheCon Europe.

Photo: Jan Michalko / plain schwarz

These were my very first ApacheCon experiences, and it was great to be part of the community in this memorial year - yes, it's the 20th anniversary of Apache Software Foundation.

A key message I've got from the conferences is about the power of community. Of course, all sessions are technically stimulating and exciting to learn, but, more than that, I was impressed by how to make such a big conference possible as a result of gathering diverse people and projects from the OSS community.

ApacheCon North America @ Las Vegas

In this annual conference, we have presented the recent updates on the Hivemall project:

I feel the audiences had a highly practical point of view; through the questions, they were trying to get a deeper understanding of how Hivemall runs with MapReduce, Tez, and Spark.

I Stopped Drinking Alone for 4 Months

Sun, 08 Sep 2019 00:00:00 +0000

From March 2019 to July 2019, I experimentally stopped drinking alcohol alone. A key question was: "Do I become more productive and healthy?"

The answer is actually "NO" in terms of both; I didn't observe any significant differences, but it was an interesting opportunity to make a small change in my life.

Rule

The rule is simple: Do not drink alcohol alone anywhere, and anytime.

I was not trying to stay away from any drinking activities. Drinking with somebody else like friends and colleagues is fine, but I just shouldn't be alone.

I regularly ‐ a couple of days per week ‐ enjoy drinking alone at a bar or home. It makes me more comfortable and relaxed, and sometimes the bars become a good place for reading books, writing articles or code, and even for working in some special cases.

Stopping the habit was not easy. I needed to completely change my mind and way of spending my time over the weekends.

Hello Netrify for Hosting a Static Site

Sat, 31 Aug 2019 00:00:00 +0000

For the recent two years, I have used GitHub Pages to publish my takuti.me site built by Hugo:

Meanwhile, this article you are reading is distributed by Netlify as of Aug 31, 2019.

The motivation of this change is simply based on my interests; while I have no strong complaints to GitHub Pages, Netlify looks nicer and fitting my use case for the following reasons.

Auto publishing

First of all, synchronization with a GitHub repository and setting up auto-publishing are super easy. It has been completed in a couple of clicks on a Netlify's configuration view.

In the case of GitHub Pages + Hugo, I needed to configure extra things like CI/CD using Travis. It is technically stimulating, but I like to stay focus more on writing articles itself rather than spending a long time for implementing and managing the build & deploy scripts. Plus, site deployment on Netlify finishes much faster than the CI scripts.

Lightning Talk about Recommender Systems in Julia at #JuliaCon 2019

Fri, 26 Jul 2019 00:00:00 +0000

I have attended and presented at JuliaCon 2019 held in Baltimore, MD, USA:

Recommendation.jl: Building Recommender Systems in Julia

Recording is available at YouTube:

The 10-min lightning talk was very short. But it was a great opportunity to introduce my Recommendation.jl package and hear the comments from the audiences. Plus, I was inspired a lot by the other sessions and nice people I've met throughout the conference.

Recommender Systems in Julia

As I spoke, since recommendation is not only about machine learning, I'm trying to keep the Recommendation.jl package as simple as possible with leaving enough space for user-side customization. This approach works particularly well for Julia because of its easiness, efficiency, and great package ecosystem.

In fact, due to the limited number of practical use cases, I'm still unsure if I created the package in the right way. However, a fact that Recommendation.jl is used in a book "Julia Programming Projects" hopefully shows a certain degree of usability:

User Modeling, Adaptation, Personalization for Marketing #UMAP2019

Sat, 13 Jul 2019 00:00:00 +0000

It has been a month since I presented at The 27th ACM Conference On User Modeling, Adaptation And Personalization (UMAP2019) held in Larnaca, Cyprus:

Zero-Coding UMAP in Marketing: A Scalable Platform for Profiling and Predicting Customer Behavior by Just Clicking on the Screen

* You can download the PDF file from HERE

You may also be interested in a following video I created during the submission process of the demo paper:

UMAP in Customer Data Platform

The accepted paper has demonstrated UMAP capabilities implemented in our Arm Treasure Data Customer Data Platform with a poster:

https://twitter.com/takuti/status/1138049394387951617

* PDF version of the poster is available at HERE

Customer Data Platform (CDP) is a new type of marketer-operated applications that allows the marketing team to obtain a deeper understanding of individual customers.

Unlike traditional Customer Relationship Management (CRM) systems, CDP application gives a more detailed view of the customers by unifying not only their static attributes (e.g., name, email address) but dynamic behaviors coming from a wide variety of online and offline data sources such as web access logs, third-party sales/marketing tools, on-premise DBs, in-store physical sensors.

TokuDB in MariaDB (on Ubuntu)

Sun, 07 Apr 2019 00:00:00 +0000

TokuDB is a high-performance, scalable storage engine for MySQL/MariaDB. Since the engine is optimized for a large volume of data, nowadays it would become a reasonable option to build an in-house big data analytics solution.

As the first step toward making sure its effectiveness and efficiency, I simply tried to enable the storage engine in MariaDB running on an AWS EC2 instance. Let me share the whole procedure so that everyone can easily deep-dive into TokuDB itself, rather than some troublesome chores.

First, create an AWS EC2 instance with a security group that accepts port=3306 inbound request, and install the components as:

sudo apt-get update
sudo apt-get install mariadb-server mariadb-plugin-tokudb

Second, un-comment plugin-load-add in /etc/mysql/mariadb.conf.d/tokudb.cnf:

[mariadb]
# See https://mariadb.com/kb/en/how-to-enable-tokudb-in-mariadb/
# for instructions how to enable TokuDB
#
# See https://mariadb.com/kb/en/tokudb-differences/ for differences
# between TokuDB in MariaDB and TokuDB from http://www.tokutek.com/

plugin-load-add=ha_tokudb.so

Meanwhile, update bind-address configured in /etc/mysql/mariadb.conf.d/50-server.cnf as follows:

# this is read by the standalone daemon and embedded servers
[server]

# this is only for the mysqld standalone daemon
[mysqld]

#
# * Basic Settings
#
user            = mysql
pid-file        = /var/run/mysqld/mysqld.pid
socket          = /var/run/mysqld/mysqld.sock
port            = 3306
basedir         = /usr
datadir         = /var/lib/mysql
tmpdir          = /tmp
lc-messages-dir = /usr/share/mysql
skip-external-locking

# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
bind-address            = 0.0.0.0

Run the server:

Publishing My Master's Thesis with Documenter.jl

Sun, 31 Mar 2019 00:00:00 +0000

When I was a master's student, I had developed Recommendation.jl, a collection of well-known recommendation techniques written in Julia:

Recommendation.jl: Building Recommender Systems in Julia

Recently I have been working on creating its documentation site by using Documenter.jl, a documentation generator for Julia. This article describes how the tool allows me to easily create the website and what I'm writing on it.

Importance of documentation for Julia packages

In fact, I do not actively add state-of-the-art recommendation techniques to the Recommendation.jl package because the package simply focuses on providing a way to test well-studied traditional methods, with leaving room for user-side customization. I personally believe everything in the package is trivial in the field of recommender systems, and hence I did not write any dedicated documentations so far.

However, even if the implemented techniques are widely known (e.g., collaborative filtering, matrix factorization), I recently realized that explicitly documenting their underlying scientific concepts and mathematical equations is beneficial to provide a better understanding of the package, especially in case of the Julia programming language.

Apache Hivemall in PySpark

Sun, 24 Mar 2019 00:00:00 +0000

Apache Hivemall, a collection of machine-learning-related Hive user-defined functions (UDFs), offers Spark integration as documented here. Now, we will see how it works in PySpark.

Note that Hivemall requires Spark 2.1+. This article particularly uses Spark 2.3 and Hivemall 0.5.2, and the entire contents are available at this Google Colabo notebook.

Installation

We do need to set up Spark and Hadoop environment first of all. For example, if you are using Colabo, follow instructions as:

!apt-get install openjdk-8-jdk-headless -qq > /dev/null
!wget -q http://mirror.reverse.net/pub/apache/spark/spark-2.3.3/spark-2.3.3-bin-hadoop2.7.tgz
!tar xf spark-2.3.3-bin-hadoop2.7.tgz
!pip install -q findspark

import os
os.environ["JAVA_HOME"] = "/usr/lib/jvm/java-8-openjdk-amd64"
os.environ["SPARK_HOME"] = "/content/spark-2.3.3-bin-hadoop2.7"

import findspark
findspark.init()

Next, download hivemall-spark2.x-0.y.z-incubating-with-dependencies.jar corresponding to your Spark version from the ASF repository:

wget -q http://mirror.reverse.net/pub/apache/incubator/hivemall/0.5.2-incubating/hivemall-spark2.3-0.5.2-incubating-with-dependencies.jar

Create Spark session

Connect to the Spark instance and start a new session:

from pyspark.sql import SparkSession

spark = SparkSession.builder.master('local[*]').config('spark.jars', 'hivemall-spark2.3-0.5.2-incubating-with-dependencies.jar').enableHiveSupport().getOrCreate()

The Hivemall .jar file is explicitly loaded from jars option, and Hive connection and their UDF support are enabled by enableHiveSupport().

Feeding User-Item Interactions to Python-Based Streaming Recommendation Engine via Faust

Mon, 14 Jan 2019 00:00:00 +0000

Streaming recommendation is one of the most challenging topics in the field of recommender systems. The task requires recommendation engine to incrementally and promptly update recommendation model as new user-item interaction comes in to data streams (e.g., click, purchase, watch). I previously studied such incremental recommendation techniques, and eventually published a Python library named FluRS to make their implementation and evaluation easier.

As I described in my research paper a couple of years ago, incremental recommender systems run in three steps:

System recommends top-N items to a user by using a production recommendation model
User may interact with one or more recommended items
Recommendation model is incrementally updated based on the observed user-item interactions

In order to provide an implementation idea of the above recommender systems, this article examines Faust, a handy stream and event processing engine for Python, in combination with FluRS. Faust allows our Python code to easily consume data streams and do something for incoming events.

pip install faust

Updating FluRS recommender from a Faust processor

Assume that a dummy Kafka topic flurs-events continuously receives MovieLens rating events represented by pairs of <user, item, rating, timestamp>. In case that those events are JSON-serialized, a Faust event processor can be defined as:

Attending MLconf SF 2018 #mlconf18

Thu, 22 Nov 2018 00:00:00 +0000

I have attended MLconf 2018 in San Francisco. Since awesome speakers came from highly recognized industrial organizations, I can confidently say that MLconf can be a great place to see industrial trends and real-world "successful" use cases.

Surprisingly (and unsurprisingly), all of the following topics were covered in this one-day conference:

Interpretability
- Saliency map for images vs. TCAV
Large-scale satellite image data collection
- Make it available for developers
Scalable ML with Amazon SageMaker
- Providing cheaper scalable solution on the cloud
- Train local states (i.e., partial models) on multiple GPU-enabled instances in parallel in the streaming, incremental fashion, and finally merge them into single shared state
- Reinventing k-means to make it scalable
Uber's NLP efforts on building AI for riders and drivers
- TF-IDF + LSA vs. CNN
ML and deep learning applications
- Geospatial processing
  - Generative address to get a better understanding of geographical characteristics from satellite images
- Healthcare
  - Baidu's efforts on DL for pathology with neural conditional random field
- Fake news detector "FakerFact"
Practical lessons on differential privacy
and more!

My favorite session was Edo Liberty's one about Amazon SageMaker:

Sending Sensor Data from Mbed Simulator to Treasure Data

Fri, 02 Nov 2018 00:00:00 +0000

Arm Pelion IoT Platform makes the Mbed ecosystem much more sophisticated and enables us to seamlessly manage IoT connectivity, device and data at a unified place:

Treasure Data: A Pelion data management service

My recent experience at Mbed Connect USA tells me how people show a strong interest in advanced big data management and analytics capability of Treasure Data (TD) as a part of the Pelion family:

https://twitter.com/takuti/status/1052270115956305920

On that point, there is a great demo that shows how to send device health data from your Mbed devices to the Treasure Data platform via WiFi network:

Once data points emitted from your devices are securely stored to the platform, you will become able to gain tons of deeper insights at scale in combination with many different types of datasets (e.g., user's demographics and behavior) loaded from a variety of data sources.

To those who still don't have a physical Mbed device (including me!), this article introduces a handy version of the Mbed-to-TD data ingestion demo running on top of Mbed simulator.

Apache Hivemall at #ODSCEurope, #RecSys2018, and #MbedConnect

Fri, 26 Oct 2018 00:00:00 +0000

Apache Hivemall is a scalable machine learning library running on top of the Hadoop ecosystem, and you can freely enjoy its functionalities in your Apache Hive or Spark environment.

Do you know Google BigQuery ML? It is a new machine learning solution that runs in their BigQuery data warehousing platform. BigQuery ML enables us to apply machine learning to our massive data by just issuing a series of queries, and what Hivemall allows us to is basically the same thing; both BigQuery ML and Hivemall introduce a new paradigm "machine learning in query language," but Hivemall is more flexible in terms of selection of platform and algorithm.

I've recently spent one of the busiest and most exciting months in 2018 to talk about Hivemall at a couple of different events, Open Data Science Conference Europe (ODSC Europe), ACM Recommender Systems (RecSys), and Mbed Connect USA, in the different contexts. Let's see what I've talked there.

Open Data Science Conference Europe @ London

As the name suggests, this conference strongly focuses on publicly available research result, practical lessons, and open-source software in the field of machine learning and data science; advertising talk from industries is basically prohibited.

Understanding Research Trends in Recommender Systems from Word Cloud

Sat, 11 Nov 2017 00:00:00 +0000

The field of recommender systems grows rapidly according to the recent development of practical intelligent systems. However, even though the field is exceptionally practical compared to the other computer-science-related topics, many researchers are actively studying recommendation techniques in their lab. Here is a question: what are the research trends in recommender systems?

I tried to understand the trends from word cloud by using abstract of papers accepted for ACM RecSys Conference, one of the biggest major conferences on recommendation systems.

Collecting abstract of accepted papers

Luckily, abstract of accepted RecSys papers are well-formatted on the web page in terms of HTML structure e.g., RecSys 2017 Accepted Contributions. So, first we collect the data in a text format by using a scraping tool, especially Scrapy in this article:

$ pip install scrapy

Using Scrapy is quite easy; what I need to do is to implement a scraping module named "Spider":

import scrapy


class RecSysSpider(scrapy.Spider):
    name = 'recsys-spider'

    def start_requests(self):
        self.abstract_index = 0

        if not hasattr(self, 'yy'):
            self.yy = '17'
        elif self.yy == '14':
            self.abstract_index = 2
        elif self.yy not in ['14', '15', '16', '17']:
            raise ValueError('Invalid year: ' + self.yy)

        url = 'https://recsys.acm.org/recsys{}/accepted-contributions/'.format(self.yy)
        yield scrapy.Request(url, self.parse)

    def parse(self, response):
        for elem in response.css('ul.accordion li'):
            yield {'abstract': elem.css('div p ::text').extract()[self.abstract_index].strip()}

This Spider module allows users to pass an argument yy to specify which year's RecSys abstract you want to get.

Deploying Static Site to GitHub Pages via Travis CI

Mon, 26 Jun 2017 00:00:00 +0000

As I repeatedly mentioned in this blog, takuti.me is currently generated by Hugo:

Manual deployment

Until quite recently, my deployment workflow was not clever; I manually followed the steps as:

Write an article under takuti.me/_content directory
Hit rake command and generate my site under the public/ directory
Commit & push takuti.me/* to repository (source), except for public/
Commit & push takuti.me/public/* to repository (GitHub Pages)

It actually works, but we can automate the workflow more aggressively...it's 2017!

Utilizing `deploy:` feature on Travis CI

So, eventually deployment becomes easier thanks to the following guide:

GitHub Pages Deployment - Travis CI

My current .travis.yml looks like:

language: c
os: osx

before_script:
  - npm install -g gulp
  - npm install
  - brew update && brew install hugo

script:
  - rake

branches:
  only:
    - master

notifications:
  email: false

deploy:
  provider: pages
  skip_cleanup: true
  local_dir: public
  github_token: $GITHUB_TOKEN # Set in travis-ci.org dashboard
  on:
    branch: master
  repo: takuti/takuti.github.io
  target_branch: master
  fqdn: takuti.me

Once you have set your $GITHUB_TOKEN to environment variables in travis-ci.org, CI on source repository automatically generates your site and pushes it to GitHub Pages repository; now what we have to do is:

Write an article under takuti.me/_content directory
Commit & push takuti.me/* to repository (source), except for public/

Neat, huh?

No-Code ML Platform for Digital Marketing

Thu, 01 Jun 2017 00:00:00 +0000

Scope

Developed a scalable, no-code machine learning platform that enables marketers to profile and predict customer behavior through an intuitive graphical interface. Democratizes access to advanced ML capabilities without requiring programming skills.

Technology

Platform: Web-based GUI for ML workflow orchestration, built on AWS infrastructure
Capabilities: Customer profiling, behavior prediction, user modeling
Scale: Designed for large-scale marketing data and operations
User Experience: Point-and-click interface eliminating coding requirements

Key Innovation

Created a production-grade platform that bridges the gap between complex machine learning algorithms and business users, enabling marketing teams to leverage advanced customer analytics without technical expertise. The system handles end-to-end ML workflows from data ingestion to prediction through visual configuration.

Publication & Demo

UMAP 2019: Zero-Coding UMAP in Marketing: A Scalable Platform for Profiling and Predicting Customer Behavior by Just Clicking on the Screen [Paper] [Poster] [Video]

Impacts

Delivered the product feature in 3 months with a 5-person team. As an add-on feature of the enterprise SaaS, the system has been used by dozens of enterprise clients across Asia, Europe, and the Americas, where I took a pivotal role for stakeholder communication, user onboarding, and feature improvement.
Demonstrated at the academic conference (UMAP 2019), showcasing how machine learning can be made accessible to non-technical users while maintaining scalability and sophistication in customer behavior modeling.

Hugo meets kramdown + KaTeX #gohugo

Sun, 28 May 2017 00:00:00 +0000

Recently, math rendering library in this page, takuti.me, has been switched from MathJax to KaTex to improve performance. You can check KaTeX's fast, beautiful rendering in the following articles:

This article describes how I have accomplished the modification.

The Markdown + LaTeX syntax issue

I started using Hugo 1+ years ago, and ever since then, this blog has utilized kramdown & MathJax, one of the best combinations that allows us to naturally integrate Markdown and LaTeX syntax. Following article describes the details of motivation behind the choice: Migrate to Hugo from Jekyll: Another Solution for the MathJax+Markdown Issue.

As I discussed in the article, integrating Markdown and LaTeX syntax is essentially hard problem for static site generators, because _ (underscore) plays an important role on the both syntax; many Markdown parsers (including Hugo's default Markdown processor Blackfriday) mistakenly understand LaTeX's _ (for subscripts) as an indicator of italic.

Moving to GitHub Pages

Sun, 30 Apr 2017 00:00:00 +0000

takuti.me is currently generated by Hugo, a fast static site generator written in Go, as I introduced a year and a half ago.

Until this point, this site initially started as a WordPress blog in 2012, and I switched to create the contents by using a static site generator, Jekyll, from 2014 to 2015.

In fact, all of the WordPress blog, Jekyll and Hugo sites were hosted on VPS having the basic CentOS + nginx (or Apache) environment. However, last week I eventually moved the site to GitHub Pages.

Basically, there were only 2 steps I needed to undergo:

Create my github.io page and upload contents
Set up custom HTTPS domain https://takuti.me/

(Nothing has changed from a user side.)

Surprisingly, moving a VPS-hosted site to GitHub Pages was really easy and comfortable than I expected for a long time. This new environment clearly motivates my weekly writing activity :)

Below I briefly writes down what I did to move this site.

Step 1: Hosting Static Pages Generated by Hugo on GitHub Pages

In order to host your Hugo-generated site on GitHub Pages, you first need to create own github.io space as explained here: GitHub Pages Basics. Moreover, Hugo officially provides a tutorial page: Hosting on GitHub.

Comparison of Running Time of Cached/Uncached Spark RDD

Sun, 16 Apr 2017 00:00:00 +0000

Resilient Distributed Dataset (RDD) is a distributed parallel data model in Spark. The model enables us to think of our distributed data like a single collection. In this article, I introduce some basics and show experimental result which clearly demonstrates the strength of RDD.

First and foremost, there are two different types of operations for RDD: transformation and action.

Type I: Transformation

Transformation corresponds to Scala transformers such as map() and filter(); we can apply both map() and filter() operations for RDDs in a similar way to the standard Scala collections.

In the Scala collections, this kind of operations return a new collection as:

scala> Seq(1, 2, 3).map(_ * 10)
res: Seq[Int] = List(10, 20, 30)

In RDDs, transformation similarly behaves; a new RDD will be returned as a result of transformation.

However, there is a huge difference between Scala collections and RDDs: result of transformation is NOT immediately computed. That is, transformation stands on laziness.

In case that we have a RDD which contains same values as the above example, rdd.map() does not return any "collections":

Parallel Programming vs. Concurrent Programming

Sat, 25 Feb 2017 00:00:00 +0000

What is the difference between parallel programming and concurrent programming? There is a lot of definitions in the literature.

"Executing simultaneously" vs. "in progress at the same time"

For instance, The Art of Concurrency defines the difference as follows:

A system is said to be concurrent if it can support two or more actions in progress at the same time. A system is said to be parallel if it can support two or more actions executing simultaneously. The key concept and difference between these definitions is the phrase "in progress."

This definition says that, in concurrent systems, multiple actions can be in progress (may not be executed) at the same time. Meanwhile, multiple actions are simultaneously executed in parallel systems. In fact, concurrency and parallelism are conceptually overlapped to some degree, but "in progress" clearly makes them different.

Even though such definition is concrete and precise, it is not intuitive enough; we cannot easily imagine what "in progress" indicates.

Recently, I am taking an online course Parallel Programming on Coursera, and the course gave me more intuitive and easy-to-understand definition of parallel and concurrent programming as follows.

FluRS: A Python Library for Online Item Recommendation

Sat, 21 Jan 2017 00:00:00 +0000

Last week, I introduced a Julia package for recommender systems: Recommendation.jl: Building Recommender Systems in Julia.

However, its functionality is still low, and I argued that implementing more powerful recommendation techniques and update() function is important. Thus, this article provides FluRS, another open-sourced library for recommendation. Unlike Recommendation.jl, this recommender-specific library is written in Python from a practical point of view.

The initial version (v0.0.1) of FluRS is already published to PyPI. You can install and use the library by:

$ pip install flurs

This article simply describes basic ideas and concepts behind the implementation. See the README file to learn how to use the library.

Python for recommendation

In a context of recommendation, one of the most popular open-source libraries written in Python is fastFM, a library for factorization machine (FM) which is a state-of-the-art flexible factorization model. However, fastFM is actually a more generic library for FM-based prediction; it can be used for various applications such as, but not limited to recommendation.

Recommendation.jl: Building Recommender Systems in Julia

Sat, 14 Jan 2017 00:00:00 +0000

I have recently published Recommendation.jl, a Julia package for recommender systems. The package is already registered in METADATA.jl, so it can be installed by:

$ julia
julia> Pkg.add("Recommendation")

Last year, I took Introduction to Recommender Systems, an online course created by University of Minnesota, on Coursera. Although the course assignments originally require us to use spreadsheet (on Google Drive or MS Excel), I personally solved all of them by programming in Julia. Recommendation.jl is created as a result of the coursework assignments.

This article explains about the details of Recommendation.jl. If you are interested in functionality and usage, you can refer to README of the repository. In case you have comments or suggestions for future update, issues and/or pull requests are very welcome!

Note: The online course is now opening as Recommender Systems Specialization. Course contents might be different from the previous version that I have completed.

Why Julia?

Julia is a relatively new programming language developed by computer scientists at MIT, and the language mainly focuses on high-performance scientific computing by utilizing the just-in-time compiler. Conventionally, MATLAB has been widely used for numerical computing, but it is in some sense inefficient proprietary software. Thus, open-sourced Julia's efficient implementation is getting the attention of research communities in these days: Julia: A Fast Language for Numerical Computing.

My New Year's Resolution 2017: Write an Article Every Week

Sat, 07 Jan 2017 00:00:00 +0000

In this year, I focus on writing more and more articles on this blog. In particular, I write an article every week; 52 articles should be published until the end of 2017. Each post probably describes what I have created, read, and experienced in a week. Note that the articles will be written both in Japanese and English depending on the contents.

Most importantly, taking full advantaged of a personal publishing tool is one of the best ways to enrich the quality and quantity of your portfolio. Since we cannot write anything without a certain result, learning and creating something (e.g., writing code, reading books, taking online courses) should be essential to write 52 articles. Your portfolio is definitely matured by the efforts made by yourself.

OK, I personally like to enhance my portfolio for the following reasons.

First of all, my career as a professional engineer starts from this April after graduation from the master's course. In fact, I have enjoyed programming and engaged in computer science for more than 5 years, but everything what I have experienced before was recognized as student's results. In contrast to the students' immature results, recognition for professional work is strict; I cannot be distinguished unless I demonstrate my skills in some way. Hence, I expect this blog to be a certificate of my professional skills.

Recommendation.jl

Sat, 01 Oct 2016 00:00:00 +0000

Scope

Created Recommendation.jl, a Julia package for building recommender systems that leverages Julia's high-performance scientific computing capabilities. Designed with extensibility in mind through separated data access, algorithm, and recommender layers.

Technology

Language: Julia
Algorithms: k-Nearest Neighbors, Matrix Factorization, Non-personalized Baselines
Evaluation: Recall, Precision, and other dedicated recommendation metrics
Architecture: Flexible three-layer design (data access, algorithm, recommender)

Key Features

High-Performance Computing: Leverages Julia's efficiency for scientific computing in recommendation algorithms
Easy Experimentation: Enables quick implementation and testing of recommender systems
Extensibility: Separated layers allow users to build custom recommendation models with minimal effort
Comprehensive Toolkit: Includes algorithms, evaluation metrics, and baseline methods

Presentation

JuliaCon 2022: Recommendation.jl: Modeling User-Item Interactions in Julia [Slides] [Video]
JuliaCon 2019: Recommendation.jl: Building Recommender Systems in Julia [Slides] [Video]

Learn More

Package: Recommendation.jl
Articles: Building Recommender Systems in Julia

Hivemall

Thu, 01 Sep 2016 00:00:00 +0000

Scope

Core contributor to Hivemall, a scalable machine learning library that enables SQL-based machine learning at scale on Apache Hive, Spark, and Pig. Contributed to new algorithms, PySpark integration, and community outreach.

Technology

Platform: Apache Hive, Apache Spark (PySpark), Apache Pig
Algorithms: Classification, Regression, Recommendation, Anomaly Detection, NLP, Topic Modeling
Features: Field-Aware Factorization Machines, Data Sketching, Feature Engineering UDFs
Integration: XGBoost, LightGBM, Digdag workflow engine

Key Contributions

PySpark Integration: Enabled seamless access to Hivemall capabilities through SparkSession with Hive support, combining Spark/Hive scalability with Python ecosystem flexibility
Algorithm Implementation: Contributed state-of-the-art generalized factor models and recommendation techniques
Query-Based ML in Production: Simplified end-to-end ML workflows through SQL-like interface, reducing complexity from numerous code fragments to dozens of lines of queries

Presentations & Publications

ApacheCon Europe 2019: Apache Hivemall Meets PySpark [Slides] [Video]
ApacheCon North America 2019: What's New and Coming to Apache Hivemall (v0.5.2-incubating features and roadmap) [Slides]
RecSys 2018: Query-Based Simple and Scalable Recommender Systems with Apache Hivemall [Paper] [Poster] [Video]
ODSC Europe 2018: Apache Hivemall: Query-Based Handy, Scalable Machine Learning on Hive [Slides] [Video]

Learn More

Project: Apache Hivemall (incubating)
GitHub: apache/incubator-hivemall
Production: No-Coding ML Platform for Marketing

Migrate to Hugo from Jekyll: Another Solution for the MathJax+Markdown Issue

Mon, 19 Oct 2015 00:00:00 +0000

Migrate to Hugo from Jekyll

I had used Jekyll to generate this site for 1+ years. However, recently generating site took more than 10 seconds due to increasing site contents; this was really stressful when I like to check the generated site continuously (e.g. style updating).

Hence, from this article, I have migrated to Hugo, and this fast static site generator can build my site less than 1 second! Migrating to Hugo from Jekyll is not so difficult because there are many useful information on the Internet such as:

In particular, spf13/spf13.com, a repository of Hugo creator's website, is practically useful to understand what we have to do.

MathJax+Markdwon issue

In the TeX syntax, we frequently use _ (underline) to render subscripts. However, at the same time, underline also indicates italic font in Markdwon. This situation raises a mis-converting issue; when Markdown is first parsed, MathJax might not understand TeX formulas correctly.

PyCon JP 2015 #pyconjp

Tue, 13 Oct 2015 00:00:00 +0000

Following phpcon2015, I have attended PyCon JP 2015 at the end of last week.

Although I have more than 8-year programming experience, I first write Python code only a year ago. However, despite the really short-term experience, now Python is definitely one of my most favorite programming languages. I use Python in many different purposes such as research, private projects, part-time job, and competitive programming.

My usage scenes of Python is really wide-ranging, right? Actually, the theme of PyCon JP 2015 was Possibilities of Python, and everyone agrees with its possibilities. Also, the conference schedule clearly illustrates possibilities of Python; we can see a wide variety of talks: hardware-related (robotics, FPGA), web development, data science and machine learning.

Due to too much conference contents, summarizing them one-by-one is hard for me. So, I just list some GitHub repositories based on my stars during the conference:

getsentry / sentry
- Keynote talk focused on Python technologies around error/crash reporting.
- Highly practical and interesting keynote.
JukkaL / mypy
python / typeshed
- Talk about Python 3.x and type hints (from 3.5).
- I still use Python 2.7, but this talk made me want to use 3.x.
google / protobuf
grpc / grpc
- Talk about gRPC and Kubernetes by Googler.
- Good introduction to modern infrastructure, including concepts of containers.
mocobeta / janome
- Japanese morphological analyzer in Python.
- Easy to follow the talk about how Japanese morphological analyzer works.
renyuanL / pythonTurtleInChinese
- Translate Python code into Chinese like for i in 範囲(100): ("範囲" means "range" in English)
- Thought-provoking talk on programming education.
c-bata / pandas-validator
blaze / dask
fabric / fabric
mitsuhiko / click
airtoxin / pysqldf
- Lightning talks by 10 speakers.
- Some of them introduced very useful tools.
- Especially, since I became curious about command-line tool development with Click, I have tried it by creating tiny twitter client after the talk.

Of course, there were many other interesting talks. In particular, presentations about tweet analysis, semantic web, pandas internals and ad science were good because these topics were very close to my current interests.

Japan PHP Conference 2015 #phpcon2015

Sun, 04 Oct 2015 00:00:00 +0000

I have attended phpcon2015, Japan PHP Conference 2015, on October 3.

As you know, this year is very important for PHP users, because PHP7 will be released very soon. So, the theme of this PHP conference is 7.

Though this was my first experience of attending large-scale conference in a cool geek community, I really enjoyed many exciting talks. Basically, I listened invited foreign speakers' talk, and all of them provided super favorable knowledge.

"Performance Testing Modern Apps" @dustinwhittle

He introduced many useful performance testing tools such as:

"Automate performance testing, and treat performance as a feature"

"From PHP to Machine Code" @juokaz

Next, this was great introduction to how programming languages work (not only PHP). Today, a wide variety of engineers/designers use PHP, and some of them actually do not have computer science background. Hence, this kind of fundamental knowledge is valuable.

Also, we were really surprised his latest work in Python, called PyHP, to implement PHP with JIT support.

Machine Learning Summer School 2015 Kyoto #MLSSKYOTO

Sat, 03 Oct 2015 00:00:00 +0000

Hi, I am takuti, a master's student in Japan. Currently, I am working on matrix factorization and approximation. Also, my research interests are in web engineering, mining and their applications such as recommender systems.

I have attended to Machine Learning Summer School 2015 Kyoto (MLSS'15) from August 23 to September 4. This entry briefly reviews each of 14 exciting lectures in the summer school. Note that there might be mistakes in the content. In that case, please let me know via comments or twitter.

Slides for the lectures are available at: http://www.iip.ist.i.kyoto-u.ac.jp/mlss15/doku.php?id=schedule

TL;DR

https://twitter.com/takuti/status/636907107107803137

(with strong emphasis on regularization and LASSO)

MLSS Sensu

Convex Optimization

Stephen P. Boyd, Stanford

This is one of the most impressive lectures in MLSS'15 because Prof. Boyd provided us really interesting and stimulating lecture; it was like a show. He interactively discussed key ideas and specific applications of convex optimization problems. Also, thanks to Convex Optimization Short Course, we can easily learn and try to solve real convex optimization problems on our laptop after MLSS.

How to Derive the Normal Equation

Tue, 21 Apr 2015 00:00:00 +0000

In the linear regression tasks, the normal equation is widely used to find optimal parameters. However, Pattern Recognition and Machine Learning (RPML), one of the most popular machine learning textbooks, does not explain details of the derivation process. So, this article demonstrates how to derive the equation.

Linear regression model

We define linear regression model as:

$
y = \textbf{w}^{\mathrm{T}}\phi(\textbf{x})$

for a input vector $\textbf{x}$, base function $\phi$ and output $y$.

The main task is to find an optimal parameter $\textbf{w}$ from $N$ learning data sets, $(\textbf{x}_1, t_1), (\textbf{x}_2, t_2), \ldots, (\textbf{x}_N, t_N)$. As a result of such learning step, we can predict output for any input $\textbf{x}$.

Least squares method

How can we estimate an optimal parameter $\textbf{w}$? The answer is quite simple: minimization of the total prediction error. When we already have parameters, the total prediction error for the $N$ learning data may be computed by $\sum_{n=1}^{N} (t_n-\textbf{w}^{\mathrm{T}}\phi(\textbf{x}_n))$. Is it correct?

Unfortunately, this formula has two problems. First, if learning data such that $t_n-\textbf{w}^{\mathrm{T}}\phi(\textbf{x}_n)< 0$ exists, above formula does not represent "total error". Second, since the formula is linear for $\textbf{w}$, we cannot minimize it. Thus, squared error function $E(\textbf{w})$ is considered as:

Hello English Entries

Sun, 09 Nov 2014 00:00:00 +0000

Hi, I'm takuti.

Until now, this blog had only Japanese articles. However, from this post, blog.takuti.me switches to an English blog. Previous Japanese articles are available here continuously, but I will not write any more Japanese posts. If you want to read my Japanese, you can see it on my hatenablog, my Japanese blog on one of the most popular blog services in Japan.

Renewed blog.takuti.me may cover following topics:

Several algorithms in Computer Science field (and their implenetations).
Applications developed by me.
Books and web sites which are technologically/scientifically stimulating.

Currently, you can see my implementations and specific applications on my github repositories. On this blog, I will explain about them more deeply.

Please send your comments freely! Everything is welcome. Thank you.

P.S. (Apr. 18, 2015)

I said "I will not write any more Japanese posts" in this entry, but I will change the opinion. I use this blog as a bilingual blog; more specifically, I will write articles both in English and Japanese.

Streaming Recommendation Algorithms

Wed, 01 Jan 2014 00:00:00 +0000

Scope

Developed novel recommendation algorithms and a Python library to handle dynamic user-item interactions in real-time streaming scenarios, addressing the cold-start problem and enabling efficient online learning.

Technology

Algorithms: Factorization Machines, Sketching-based Online Learning, Relational Clustering
Implementation: Python, NumPy, SciPy
Use Cases: E-commerce recommendations, social networking, folksonomies

Key Innovations

Incremental Factorization Machines: Persistently handles cold-starting items in online recommendation scenarios
Dynamic Sketching: Efficiently captures and adapts to changing user-item interactions in streaming data
FluRS Library: Python library providing flexible, efficient streaming recommendation algorithms with dependency injection design pattern

Publications & Presentations

EuroSciPy 2017: FluRS: A Library for Streaming Recommendation Algorithms [Video] [Slides]
CHIIR 2017: Sketching Dynamic User-Item Interactions for Online Item Recommendation [Paper] [Poster]
RecProfile 2016: Incremental Factorization Machines for Persistently Cold-starting Online Item Recommendation [Paper] [Slides]
WIMS 2015: User Modeling in Folksonomies: Relational Clustering and Tag Weighting [Paper] [Slides] [Code]
- Not exactly a "streaming" recommender, but it was a foundational study surfacing the limitations of batch algorithms.

Learn More

Code: stream-recommender (Research implementations)
Library: FluRS (Production-ready Python library)
Article: FluRS: A Python Library for Online Item Recommendation

Takuya Kitazawa

Act on Complicated Things

Complex vs. Complicated

What I'm Doing Now

Search for a new (remote) opportunity

5 months of practicing French

A Recipe for Sustainable Data Products

Toward Inclusive Education

Who is Digitally Prepared?

Task-based assessment

Professional Services

Product Development

Digital Transformation

Digital Inclusion and Participation

Mentorship

The AI Divide

Why Offline Learning Still Matters in 2026

Being online is a privilege

Hands-on needs are in the field

Missing Link to Bridge the Digital Divide

"We," Not "I"

Power of Collective Actions

Uche Safe App

Scope

Key Features

Testimonial

Contact Me

The End of the Beginning—What I Talk About When I Talk About Malawi

Materializing Digital Transformation

I'm Not "China"—Connecting HERE and THERE in Systems

Language: Behind the Power Dynamics in Information Society

Languages represent the power

One of the Poorest Life Is Not *That* Bad

Imbalance between earning and spending

Relativize Malawi, and Rethink Their Contexts

My 2023 Annual Review: Between Individual and Systems

Is Computer Education Always Good?

Starting with Humanity—Growth and Technology Otherwise Hurt You

Dilemma over "Best Practice": How We Could Develop Data Protection Practices in Malawi

The basics

Definition, Role, and Current Status of Digital Literacy in Malawi #LiteracyDay

TL;DR

Starting Field Study on How Information Flows in Malawi

Digital Transformation in Malawi

Scope

Technology

Key Contributions

Testimonial

Runner's Search for Identity

Information Sustainability, Mindful Consumption, and Healthy Engineering

When We Lose Autonomy—Whose Life Are You Living?

Autonomy brings meaning to life

Three Perspectives on Large Language Models and Their Emerging Applications

How Information Flows: From Field Studies to Risk Mitigation

The Complexity of Simplicity: Tacos, Sushi, and ChatGPT

My 2022 Annual Review: Towards the Deepness

Connecting to myself

Mentoring on @ADPList. Here's What I Think. #ADPListWrapped

Would you be willing to take an action NOW?

Data Are Created, Collected, and Processed by People

Why people matter in data science

Fluid People and Blended Society: How Systems Model "Dividuals"

There is no such a thing as an "individual"

The Locality of Information and Technology

The driving force behind the world

Google Cloud Dataflow and Its Positive Impact on Developer Productivity

Programming model as an implicit enforcement mechanism

Connecting the Dots in Complexity

Updating My View of Nuclear Power

Challenges

Why We "Productize"

Gradient of offering

Ethics in Recommendation Pipeline—A First Look at RecSys 2022 Papers

My First Month as Self-Employed

What are you doing now?

Reviewing Things I Do Not Know About the Indian Act #NDTR

Why Your Job Title Matters (Cont.)─Technology for the People

What makes "product" so weird

Seeing Past and Present in Coursera "Machine Learning on Google Cloud" Specialization

The basics

One of the Poorest Life Is Not That Bad

Save HTML

Hi Product Managers, Are You Creating Products That You Love?