Learning-based Models for Intelligent Decision Systems with Application to Crowdsourced Systems and Healthcare

Abstract

Technologies like artificial intelligence, the Internet of Things, quantum computing, roboticprocess automation, and genomics have ushered in the fourth industrial revolution. Digital (crowdsourcing) platforms are one of the technologies driving the fourth industrial revolution due to their new capabilities. Rather than relying on connections forged in the physical world, digital crowdsourcing platforms enable businesses to connect with users online and gather data that helps them improve their service offerings. A digital crowdsourcing platform is a network that expedites valuable connections, interactions, and exchanges between people. These exchanges include sharing information, selling products, and offering services. Seven of the top ten most valuable companies in the world are now based on a platform business model, and according to a McKinsey research report, more than 30% of global economic activity could be mediated by digital crowdsourcing platforms by 2025 (Schenker [2019]). Well-known crowdsourcing platforms like Uber, Airbnb, Amazon, StackOverflow, YouTube, Facebook, and GrubHub are so successful because they save users time and money, while also eliminating traditional industry brokerages and providing potential for people to profit from their skills. Therefore, given the positive economic and environmental impacts of crowdsourcing platforms, it is important to study how their pricing policy works. Our business partner in this study, Convoy, is a digital freight network transporting goods around the world with endless capacity (number of carriers in the network) and zero waste (eradicating empty miles that happens when trucks do not have any cargo in the back) by eliminating carbon emissions from our planet. In other words, Convoy operates a digital marketplace for shippers and carriers, and it is like Uber for trucking. Convoy uses digital crowdsourcing technology to make freight more efficient, reduce shippers’ costs, and increase carriers’ earnings. Convoy uses technology to solve waste and inefficiency problems in the $800B trucking industry, which generates more than 87M metric tons of wasted CO2-equivalent emissions from empty trucks/miles (Convoy [2022]). As more carriers join the Convoy’s network, capacity increases and shippers see lower prices per mile and higher quality. A common theme in crowdsourcing platforms including Convoy is that the platform has a limited budget and a set of jobs (shipments in our case) to accomplish by a pool of online workers (drivers on Convoy app). A key point to making these crowdsourcing platforms efficient is to design proper incentive pricing policy for workers. The tradeoff between efficiency and workers’ incentives makes the pricing decisions complex in these platforms. In a realistic crowdsourcing setting, each worker has specific preferences which define the set of jobs the worker willing to do. For instance, based on driver’s interest in shipment millage, location, time, number of stops, and price the driver can only accomplish some shipments, and not all. Note that the platform has a limited budget and should focus on picking the right driver and offering the right price. The focus of our study is on designing pricing mechanisms (dynamic and auction) for empty trucks. In this thesis, we design pricing mechanisms for the following setting: there is a digital platform who has a set of jobs (shipments) and a limited budget. To do the shipments, there is a pool of drivers. Each driver has certain interests which makes the driver eligible to do only certain shipments, and not all. Moreover, each driver has a (private) minimum cost (unknown to the driver), which is the amount the driver is willing to gain for accomplishing a shipment. We assume that a driver can do only one job (one or multiple shipments). Dynamic pricing is a pricing strategy based on real-time market demand and supply data that applies variable prices instead of a fixed price. There are often uncertainties during pricing decision making, such as unknown demand. Firms aim to optimize their revenue while learning demand. Therefore, another critical aspect of pricing is understanding customer decision-making patterns. One of the undesirable effects of dynamic pricing is that it can lead to a type of customer behavior that is referred to as a strategic behavior (due to human rationality) in the revenue management literature (Tang and Netessine [2009]). Strategic customers decide not only which products to purchase but also when to purchase them (at which price). Therefore, a potential drawback of dynamic pricing is that strategic customers contribute to delay in sales. Demand uncertainty in dynamic pricing causes an inherent trade-off between exploration and exploitation. The exploration involved in learning demand and suboptimal prices and the exploitation which is the price that appears to be the best price so far occurs due to cost minimization. In general, exploitation occurs due to revenue maximization. One way of modeling this exploration-exploitation trade-off is using a classic reinforcement learning framework called a multi-armed bandit modeling problem. Multi-armed bandit (MAB) problems are powerful frameworks for algorithms to make sequential decisions under uncertainty with an exploration-exploitation trade-off and are well-known models in reinforcement and online learning. There are three essential variants of the MAB problems: stochastic bandits, adversarial bandits, and Markovian bandits. Many algorithms have been proposed to solve these three types of MAB problems. Our focus in Chapter 2 is on the Upper Confidence Bound (UCB) algorithm, which is a deterministic algorithm that focuses on exploration-exploitation based on a confidence boundary. There are different variants of the UCB algorithms, such as UCB1, UCB2, KL-UCB, lil’ UCB, etc. In Chapter 2, we choose the UCB1 algorithm, which selects the best arm based on an optimistic estimate and achieves regret that increases only logarithmically with the number of actions taken. Regret in MAB problems is defined as the expected loss in each time step due to not playing the optimal arm. The objective of our study in this chapter is to propose an optimal incentive (UCB-based) dynamic pricing mechanism to reduce an imbalance problem in a trailer-sharing system subject to a budget limitation. There is a basic pricing mechanism known as the posted price that a firm chooses a price and offers to customers, take-it-or-leave-it. This mechanism is transactionally efficient (because the posted price is the platform’s best and final offered price and there is no negotiation), easy to communicate, and can be executed quickly. However, it burdens the firm with setting the right price based on customers’ willingness-to-pay and the firm’s supply. There are other trends in pricing mechanisms, such as an auction and haggling, in which the firm allows customers to decide on the price by submitting their bids (negotiating the price). Although these pricing mechanisms are transactionally more burdensome, they simplify willingness-to-pay discovery and potentially increase revenue. Wang [1993] analyzes when an auction pricing mechanism is a better selling mechanism than posted price mechanism. He demonstrates the auction selling revenue dominates a posted price when the characteristics of the distribution of the bidder’s value are heterogeneous. For example, Mukhopadhyay and Kekre [2002] demonstrate the use of an auction pricing mechanism to find the lowestcost supplier all over the world (see more in Brown et al. [2010]). A well-designed auction pricing mechanism causes the final price to reflect the willingness to-pay or true preference. With modern technology and algorithms, an auction pricing mechanism can be implemented quickly and is superior at generating revenue in a digital environment relative to a posted price mechanism (Cachon [2020]). Our objective in Chapter 3 is to design an optimal incentive auction pricing mechanism to maximize the cumulative profit via empty trailer return in a trailer-sharing system in the presence of rational drivers. We also compare the performance of our auction pricing mechanism with our dynamic posted price mechanism, which is measured in terms of “regret.” Finally, we study using natural language processing (NLP) and machine learning in healthcare. The use case that interests us is the application of a specific form of NLP, namely Named Entity Recognition (NER), to automate data extraction from clinical records. New tools are being developed to speed up manual extraction by using NER, which, when properly trained, allows a model to automatically identify and extract relevant data from a body of text. In this study, we evaluate the effectiveness of using Amazon Comprehend Medical (ACM) in assessing a cohort of patients with multiple myeloma seen at the Seattle Cancer Care Alliance (SCCA). This tool uses NER to extract relevant data from unstructured texts and compile them into datasets. More specifically, it recognizes a feature in a document, extracts it, and classifies it as a drug, symptom, diagnosis, or another appropriate classification. It also records the location of the feature in the clinical records. We compared the NER features to manually extracted data to explore how the two sources complement each other and to evaluate the accuracy of NLP algorithm in ACM. We focus on ACM applications for multiple myeloma for this study. Multiple myeloma is a form of cancer that affects bone marrow plasma cells and currently has no known cure. In the U.S., there were around 35, 000 new cases in 2021 with a 5-year relative survival rate of 55% between 2011 and 2017, making it the second most common type of blood cancer and 14th most common cancer overall. Researchers continue to investigate the most effective ways to treat multiple myeloma, and current treatments can limit the spread of the disease and sometimes cause complete remission. We chose this disease because of the substantial data we have able to collect as a result of the patients’ relatively long survival time, the regular monthly follow-ups, and the SCCA’s robust data due to being one of the leading bone marrow transplant centers in the country.