Sejnowski, T. J. (2018). The Deep Learning Revolution. Cambridge, MA: The MIT Press.
How deep learning―from Google Translate to driverless cars to personal cognitive assistants―is changing our lives and transforming every sector of the economy.
The deep learning revolution has brought us driverless cars, the greatly improved Google Translate, fluent conversations with Siri and Alexa, and enormous profits from automated trading on the New York Stock Exchange. Deep learning networks can play poker better than professional poker players and defeat a world champion at Go. In this book, Terry Sejnowski explains how deep learning went from being an arcane academic field to a disruptive technology in the information economy.
Sejnowski played an important role in the founding of deep learning, as one of a small group of researchers in the 1980s who challenged the prevailing logic-and-symbol based version of AI. The new version of AI Sejnowski and others developed, which became deep learning, is fueled instead by data. Deep networks learn from data in the same way that babies experience the world, starting with fresh eyes and gradually acquiring the skills needed to navigate novel environments. Learning algorithms extract information from raw data; information can be used to create knowledge; knowledge underlies understanding; understanding leads to wisdom. Someday a driverless car will know the road better than you do and drive with more skill; a deep learning network will diagnose your illness; a personal cognitive assistant will augment your puny human brain. It took nature many millions of years to evolve human intelligence; AI is on a trajectory measured in decades. Sejnowski prepares us for a deep learning future.
Buzzwords like “deep learning” and “neural networks” are everywhere, but so much of the popular understanding is misguided, says Terrence Sejnowski, a computational neuroscientist at the Salk Institute for Biological Studies.
Sejnowski, a pioneer in the study of learning algorithms, is the author of The Deep Learning Revolution(out next week from MIT Press). He argues that the hype about killer AI or robots making us obsolete ignores exciting possibilities happening in the fields of computer science and neuroscience, and what can happen when artificial intelligence meets human intelligence.
Machine learning is a very large field and goes way back. Originally, people were calling it “pattern recognition,” but the algorithms became much broader and much more sophisticated mathematically. Within machine learning are neural networks inspired by the brain, and then deep learning. Deep learning algorithms have a particular architecture with many layers that flow through the network. So basically, deep learning is one part of machine learning and machine learning is one part of AI.
December 2012 at the NIPS meeting, which is the biggest AI conference. There, [computer scientist] Geoff Hinton and two of his graduate students showed you could take a very large dataset called ImageNet, with 10,000 categories and 10 million images, and reduce the classification error by 20 percent using deep learning.Traditionally on that dataset, error decreases by less than 1 percent in one year. In one year, 20 years of research was bypassed. That really opened the floodgates.
The inspiration for deep learning really comes from neuroscience.
AlphaGo, the program that beat the Go champion included not just a model of the cortex, but also a model of a part of the brain called the basal ganglia, which is important for making a sequence of decisions to meet a goal. There’s an algorithm there called temporal differences, developed back in the ‘80s by Richard Sutton, that, when coupled with deep learning, is capable of very sophisticated plays that no human has ever seen before.
there’s a convergence occurring between AI and human intelligence. As we learn more and more about how the brain works, that’s going to reflect back in AI. But at the same time, they’re actually creating a whole theory of learning that can be applied to understanding the brain and allowing us to analyze the thousands of neurons and how their activities are coming out. So there’s this feedback loop between neuroscience and AI
meetings with Chief Learning Officers, talent management leaders, and vendors of next generation learning tools.
The corporate L&D industry is over $140 billion in size, and it crosses over into the $300 billion marketplace for college degrees, professional development, and secondary education around the world.
Digital Learning does not mean learning on your phone, it means “bringing learning to where employees are.” In other words, this new era is not only a shift in tools, it’s a shift toward employee-centric design. Shifting from “instructional design” to “experience design” and using design thinking are key here.
1) The traditional LMS is no longer the center of corporate learning, and it’s starting to go away.
LMS platforms were designed around the traditional content model, using a 17 year old standard called SCORM. SCORM is a technology developed in the 1980s, originally intended to help companies like track training records from their CD-ROM based training programs.
the paradigm that we built was focused on the idea of a “course catalog,” an artifact that makes sense for formal education, but no longer feels relevant for much of our learning today.
not saying the $4 billion LMS market is dead, but the center or action has moved (ie. their cheese has been moved). Today’s LMS is much more of a compliance management system, serving as a platform for record-keeping, and this function can now be replaced by new technologies.
We have come from a world of CD ROMs to online courseware (early 2000s) to an explosion of video and instructional content (YouTube and MOOCs in the last five years), to a new world of always-on, machine-curated content of all shapes and sizes. The LMS, which was largely architected in the early 2000s, simply has not kept up effectively.
2) The emergence of the X-API makes everything we do part of learning.
In the days of SCORM (the technology developed by Boeing in the 1980s to track CD Roms) we could only really track what you did in a traditional or e-learning course. Today all these other activities are trackable using the X-API (also called Tin Can or the Experience API). So just like Google and Facebook can track your activities on websites and your browser can track your clicks on your PC or phone, the X-API lets products like the learning record store keep track of all your digital activities at work.
3) As content grows in volume, it is falling into two categories: micro-learning and macro-learning.
4) Work Has Changed, Driving The Need for Continuous Learning
Why is all the micro learning content so important? Quite simply because the way we work has radically changed. We spend an inordinate amount of time looking for information at work, and we are constantly bombarded by distractions, messages, and emails.
5) Spaced Learning Has Arrived
If we consider the new world of content (micro and macro), how do we build an architecture that teaches people what to use when? Can we make it easier and avoid all this searching?
“spaced learning.”
Neurological research has proved that we don’t learn well through “binge education” like a course. We learn by being exposed to new skills and ideas over time, with spacing and questioning in between. Studies have shown that students who cram for final exams lose much of their memory within a few weeks, yet students who learn slowly with continuous reinforcement can capture skills and knowledge for decades.
6) A New Learning Architecture Has Emerged: With New Vendors To Consider
One of the keys to digital learning is building a new learning architecture. This means using the LMS as a “player” but not the “center,” and looking at a range of new tools and systems to bring content together.
On the upper left is a relatively new breed of vendors, including companies like Degreed, EdCast, Pathgather, Jam, Fuse, and others, that serve as “learning experience” platforms. They aggregate, curate, and add intelligence to content, without specifically storing content or authoring in any way. In a sense they develop a “learning experience,” and they are all modeled after magazine-like interfaces that enables users to browse, read, consume, and rate content.
The second category the “program experience platforms” or “learning delivery systems.” These companies, which include vendors like NovoEd, EdX, Intrepid, Everwise, and many others (including many LMS vendors), help you build a traditional learning “program” in an open and easy way. They offer pathways, chapters, social features, and features for assessment, scoring, and instructor interaction. While many of these features belong in an LMS, these systems are built in a modern cloud architecture, and they are effective for programs like sales training, executive development, onboarding, and more. In many ways you can consider them “open MOOC platforms” that let you build your own MOOCs.
The third category at the top I call “micro-learning platforms” or “adaptive learning platforms.” These are systems that operate more like intelligent, learning-centric content management systems that help you take lots of content, arrange it into micro-learning pathways and programs, and serve it up to learners at just the right time. Qstream, for example, has focused initially on sales training – and clients tell me it is useful at using spaced learning to help sales people stay up to speed (they are also entering the market for management development). Axonify is a fast-growing vendor that serves many markets, including safety training and compliance training, where people are reminded of important practices on a regular basis, and learning is assessed and tracked. Vendors in this category, again, offer LMS-like functionality, but in a way that tends to be far more useful and modern than traditional LMS systems. And I expect many others to enter this space.
Perhaps the most exciting part of tools today is the growth of AI and machine-learning systems, as well as the huge potential for virtual reality.
7) Traditional Coaching, Training, and Culture of Learning Has Not Gone Away
8) A New Business Model for Learning
he days of spending millions of dollars on learning platforms is starting to come to an end. We do have to make strategic decisions about what vendors to select, but given the rapid and immature state of the market, I would warn against spending too much money on any one vendor at a time. The market has yet to shake out, and many of these vendors could go out of business, be acquired, or simply become irrelevant in 3-5 years.
9) The Impact of Microsoft, Google, Facebook, and Slack Is Coming
The newest versions of Microsoft Teams, Google Hangouts and Google Drive, Workplace by Facebook, Slack, and other enterprise IT products now give employees the opportunity to share content, view videos, and find context-relevant documents in the flow of their daily work.
We can imagine that Microsoft’s acquisition of LinkedIn will result in some integration of Lynda.com content in the flow of work. (Imagine if you are trying to build a spreadsheet and a relevant Lynda course opens up). This is an example of “delivering learning to where people are.”
10) A new set of skills and capabilities in L&D
It’s no longer enough to consider yourself a “trainer” or “instructional designer” by career. While instructional design continues to play a role, we now need L&D to focus on “experience design,” “design thinking,” the development of “employee journey maps,” and much more experimental, data-driven, solutions in the flow of work.
lmost all the companies are now teaching themselves design thinking, they are using MVP (minimal viable product) approaches to new solutions, and they are focusing on understanding and addressing the “employee experience,” rather than just injecting new training programs into the company.
In the research project led by Ph.D. candidate Gabriel Culbertson, 48 students were recruited to play two versions of the game. In one group, students were connected via a chat interface with another player who could, if they wanted, offer advice on how to play. The second group played a version of the game in which they were definitely required to collaborate on quests.
The research group found the students in the second so-called “high-interdependence” group spent more time communicating and, as a consequence, learned more words.
The research then expanded to a larger group of 186 Reddit users who were learning Japanese. After reviewing gameplay logs, interviews and Reddit posts, they found that those who spent the most time engaged in the game learned more new words and phrases.
The Cornell research team presented its research results at the Association for Computing Machinery Conference on Human-Computer Interaction in May in San Jose, CA.
Crompton, Muilenburg and Berge’s definition for m-learning is “learning across multiple contexts, through social and content interactions, using personal electronic devices.”
The “context”in this definition encompasses m-learnng that is formalself-directed, and spontaneous learning, as well as learning that is context aware and context neutral.
therefore, m-learning can occur inside or outside the classroom, participating in a formal lesson on a mobile device; it can be self-directed, as a person determines his or her own approach to satisfy a learning goal; or spontaneous learning, as a person can use the devices to look up something that has just prompted an interest (Crompton, 2013, p. 83). (Gaming article Tallinn)Constructivist Learnings in the 1980s – Following Piage’s (1929), Brunner’s (1996) and Jonassen’s (1999) educational philosophies, constructivists proffer that knowledge acquisition develops through interactions with the environment. (p. 85). The computer was no longer a conduit for the presentation of information: it was a tool for the active manipulation of that information” (Naismith, Lonsdale, Vavoula, & Sharples, 2004, p. 12)Constructionist Learning in the 1980s – Constructionism differed from constructivism as Papert (1980) posited an additional component to constructivism: students learned best when they were actively involved in constructing social objects. The tutee position. Teaching the computer to perform tasks.Problem-Based learning in the 1990s – In the PBL, students often worked in small groups of five or six to pool knowledge and resources to solve problems. Launched the sociocultural revolution, focusing on learning in out of school contexts and the acquisition of knowledge through social interaction
Socio-Constructivist Learning in the 1990s. SCL believe that social and individual processes are independent in the co-construction of knowledge (Sullivan-Palinscar, 1998; Vygotsky, 1978).
96-97). Keegan (2002) believed that e-learning was distance learning, which has been converted to e-learning through the use of technologies such as the WWW. Which electronic media and tools constituted e-learning: e.g., did it matter if the learning took place through a networked technology, or was it simply learning with an electronic device?
99-100. Traxler (2011) described five ways in which m-learning offers new learning opportunities: 1. Contingent learning, allowing learners to respond and react to the environment and changing experiences; 2. Situated learning, in which learning takes place in the surroundings applicable to the learning; 3. Authentic learning;
Diel, W. (2013). M-Learning as a subfield of open and distance education. In: Berge and Muilenburg (Eds.). Handbook of Mobile Learning.
15) Historical context in relation to the field of distance education (embedded librarian)
16 definition of independent study (workshop on mlearning and distance education
17. Theory of transactional distance (Moore)
Cochrane, T. (2013). A Summary and Critique of M-Learning Research and Practice. In: Berge and Muilenburg (Eds.). Handbook of Mobile Learning.
( Galin class, workshop)
P 24
According to Cook and Sharples (2010) the development of M learning research has been characterized by three general faces a focus upon Devices Focus on learning outside the classroom He focus on the mobility of the learner
25
Baby I am learning studies focus upon content delivery for small screen devices and the PDA capabilities of mobile devices rather than leveraging the potential of mobile devices for collaborative learning as recommended by hope Joyner Mill Road and sharp P. 26 Large scale am learning project Several larger am learning projects have tended to focus on specific groups of learners rather than developing pedagogical strategies for the integration of am mlearning with him tertiary education in general
27
m learning research funding
In comparison am learning research projects in countries with smaller population sizes such as Australia and New Zealand are typiclly funded on a shoe string budget
28
M-learning research methodologies
I am learning research has been predominantly characterized by short term case studies focused upon The implementation of rapidly changing technologies with early adopters but with little evaluation reflection or emphasis on mainstream tertiary-education integration
p. 29 identifying the gaps in M learning research
lack of explicit underlying pedagogical theory Lack of transferable design frameworks
Pachler, N., Bachmair, B., and Cook, J. (2013). A Sociocultural Ecological Frame for Mobile Learning. In: Berge and Muilenburg (Eds.). Handbook of Mobile Learning.
(Tom video studio)
35 a line of argumentation that defines mobile devices such as mobile phones as cultural resources. Mobile cultural resources emerge within what we call a “bile complex‘, which consist of specifics structures, agency and cultural practices.
36 pedagogy looks for learning in the context of identify formation of learners within a wider societal context However at the beginning of the twentieth first century and economy oriented service function of learning driven by targets and international comparisons has started to occupy education systems and schools within them Dunning 2000 describes the lengthy transformation process from natural assets Land unskilled labor to tangible assets machinery to intangible created assets such as knowledge and information of all kinds Araya and Peters 2010 describe the development of the last 20 years in terms of faces from the post industrial economy to d information economy to the digital economy to the knowledge economy to the creative economy Cultural ecology can refer to the debate about natural resources we argue for a critical debate about the new cultural resources namely mobile devices and the services for us the focus must not be on the exploitation of mobile devices and services for learning but instead on the assimilation of learning with mobiles in informal contacts of everyday life into formal education
37
Ecology comes into being is there exists a reciprocity between perceiver and environment translated to M learning processes this means that there is a reciprocity between the mobile devices in the activity context of everyday life and the formal learning
45
Rather than focusing on the acquisition of knowledge in relation to externally defined notions of relevance increasingly in a market-oriented system individual faces the challenge of shape his/her knowledge out of his/her own sense of his/her world information is material which is selected by individuals to be transformed by them into knowledge to solve a problem in the life world
Crompton, H. (2013). A Sociocultural Ecological Frame for Mobile Learning. In: Berge and Muilenburg (Eds.). Handbook of Mobile Learning.
p. 47 As philosophies and practice move toward learner-centered pedagogies, technology in a parallel move, is now able to provide new affordances to the learner, such as learning that is personalized, contextualized, and unrestricted by temporal and spatial constrains.
The necessity for m-learning to have a theory of its own, describing exactly what makes m-learning unique from conventional, tethered electronic learning and traditional learning.
48 . Definition and devices. Four central constructs. Learning pedagogies, technological devices, context and social interactions.
“learning across multiple contexts, through social and content interactions, using personal electronic devices.”
It is difficult, and ill advisable, to determine specifically which devices should be included in a definition of m-learning, as technologies are constantly being invented or redesigned. (my note against the notion that since D2L is a MnSCU mandated tool, it must be the one and only). One should consider m-learning as the utilization of electronic devices that are easily transported and used anytime and anywhere.
49 e-learning does not have to be networked learning: therefore, e-learnng activities could be used in the classroom setting, as the often are.
Why m-learning needs a different theory beyond e-learning. Conventional e-learning is tethered, in that students are anchored to one place while learning. What sets m-learning apart from conventional e-learning is the very lack of those special and temporal constrains; learning has portability, ubiquitous access and social connectivity.
50 dominant terms for m-learning should include spontaneous, intimate, situated, connected, informal, and personal, whereas conventional e-learning should include the terms computer, multimedia, interactive, hyperlinked, and media-rich environment.
51 Criteria for M-Learning
second consideration is that one must be cognizant of the substantial amount of learning taking place beyond the academic and workplace setting.
52 proposed theories
Activity theory: Vygotsky and Engestroem
Conversation theory: Pask 1975, cybernetic and dialectic framework for how knowledge is constructed. Laurillard (2007) although conversation is common for all forms of learning, m-learning can build in more opportunities for students to have ownership and control over what they are learning through digitally facilitated, location-specific activities.
53 multiple theories;
54 Context is central construct of mobile learning. Traxler (2011) described the role of context in m-learning as “context in the wider context”, as the notion of context becomes progressively richer. This theme fits with Nasimith et al situated theory, which describes the m-learning activities promoting authentic context and culture.
55. Connectivity
unlike e-learning, the learner is not anchored to a set place. it links to Vygotsky’s sociocultural approach.
Learning happens within various social groups and locations, providing a diverse range of connected learning experiences. furthermore, connectivity is without temporal restraints, such as the schedules of educators.
55. Time
m-larning as “learning dispersed in time”
55. personalization
my note student-centered learning
Moura, A., Carvalho, A. (2013). Framework For Mobile Learning Integration Into Educational Contexts. In: Berge and Muilenburg (Eds.). Handbook of Mobile Learning.
With China muscling its way into the first ranks as a global power in science and technology—building vast new academic complexes, climbing to the top ranks of the world’s elite universities, surpassing the U.S. in PhD graduates in science and engineering, and on its way to outperforming all other nations in science and technology academic citations—I was puzzled to discover that China is on hold in offering online higher ed degrees.
To expand the nation’s technical talent pool, Chinese universities are upgrading their capacity to offer more up-to-date science and technology courses, with universities just beginning to introduce degrees in artificial intelligence, machine learning, software engineering and other advanced specialties. For China, the move is a departure from its centuries-old tradition of favoring literature and the liberal arts.
China has come a long way from cinema-style instruction to adopt more common digital learning practices, often closely following U.S. advances in online pedagogy, such as flipped classrooms and MOOCs.
Curiously, China’s reluctance to offer online degrees parallels the attitude toward online degrees in the Ivy League in the U.S.—both have embraced MOOCs while turning away from virtual degrees out of concern that remote degrees will damage their reputations.
Web 3.0 is the third generation of internet services which provide websites and applications with the technology to run. Web 3.0 is set to be powered by AI and peer-to-peer applications like blockchain. The key difference between Web 2.0 and Web 3.0 is that Web 3.0 is more focused on using innovative technologies like machine learning and AI to create more personalized content for each user. It is also expected that Web 3.0 will be more secure than its predecessors because of the system it is built upon.
Blockchains are made up of blocks that store information. Each block has a unique “hash” that differentiates it from other blocks. These blocks are then connected by a chain in chronological order. The information stored in these blocks is permanent, which makes it a very secure way to complete online transactions.
This is why cryptocurrencies, like Bitcoin, are built on blockchain technology.
Narrow AI is what we see all around us in computers today — intelligent systems that have been taught or have learned how to carry out specific tasks without being explicitly programmed how to do so.
General AI
General AI is very different and is the type of adaptable intellect found in humans, a flexible form of intelligence capable of learning how to carry out vastly different tasks, anything from haircutting to building spreadsheets or reasoning about a wide variety of topics based on its accumulated experience.
What can Narrow AI do?
There are a vast number of emerging applications for narrow AI:
Interpreting video feeds from drones carrying out visual inspections of infrastructure such as oil pipelines.
Organizing personal and business calendars.
Responding to simple customer-service queries.
Coordinating with other intelligent systems to carry out tasks like booking a hotel at a suitable time and location.
Flagging inappropriate content online, detecting wear and tear in elevators from data gathered by IoT devices.
Generating a 3D model of the world from satellite imagery… the list goes on and on.
What can General AI do?
A survey conducted among four groups of experts in 2012/13 by AI researchers Vincent C Müller and philosopher Nick Bostrom reported a 50% chance that Artificial General Intelligence (AGI) would be developed between 2040 and 2050, rising to 90% by 2075.
What is machine learning?
What are neural networks?
What are other types of AI?
Another area of AI research is evolutionary computation.
What is fueling the resurgence in AI?
What are the elements of machine learning?
As mentioned, machine learning is a subset of AI and is generally split into two main categories: supervised and unsupervised learning.
Supervised learning
Unsupervised learning
Which are the leading firms in AI?
Which AI services are available?
All of the major cloud platforms — Amazon Web Services, Microsoft Azure and Google Cloud Platform — provide access to GPU arrays for training and running machine-learning models, with Google also gearing up to let users use its Tensor Processing Units — custom chips whose design is optimized for training and running machine-learning models.
Which countries are leading the way in AI?
It’d be a big mistake to think the US tech giants have the field of AI sewn up. Chinese firms Alibaba, Baidu, and Lenovo, invest heavily in AI in fields ranging from e-commerce to autonomous driving. As a country, China is pursuing a three-step plan to turn AI into a core industry for the country, one that will be worth 150 billion yuan ($22bn) by the end of 2020 to become the world’s leading AI power by 2030.
How can I get started with AI?
While you could buy a moderately powerful Nvidia GPU for your PC — somewhere around the Nvidia GeForce RTX 2060 or faster — and start training a machine-learning model, probably the easiest way to experiment with AI-related services is via the cloud.
It requires that companies become what we call digital masters. Digital masters cultivate two capabilities: digital capability, which enables them to use innovative technologies to improve elements of the business, and leadership capability, which enables them to envision and drive organizational change in systematic and profitable ways. Together, these two capabilities allow a company to transform digital technology into business advantage.
We found that the elements of leadership capability have endured, but new elements of digital capability have come to the fore.
While strong leadership capability is even more essential than ever, its core elements — vision, engagement, and governance — are not fundamentally changed, though they are informed by recent innovations. The elements of digital capability, on the other hand, have been more profoundly altered by the rapid technological advances of recent years.
Experience design: Customer experience has become the ultimate battleground for many companies and brands.
Customer intelligence: Integrating customer data across silos and understanding customer behavior
Emotional engagement: Emotional connections with customers are as essential as technology in creating compelling customer experiences.
As ever, well-managed operations are essential to converting revenue into profit, but now we’re seeing a shift in the focus of digital transformation in this arena.
Core process automation: Amazon’s distribution centers deliver inventory to workers rather than sending workers to collect inventory. Rio Tinto, an Australian mining company, uses autonomous trucks, trains, and drilling machinery so that it can shift workers to less dangerous tasks, leading to higher productivity and better safety.
Connected and dynamic operations: Thanks to the growing availability of cheap sensors, cloud infrastructure, and machine learning, concepts such as Industry 4.0, digital threads, and digital twins have become a reality. Digital threads connecting machines, models, and processes provide a single source of truth to manage, optimize, and enhance processes from requirements definition through maintenance.
Data-driven decision-making: from backward-looking reports to real-time data. Now, connected devices, new machine learning algorithms, smarter experimentation, and plentiful data enable more-informed decisions.
Transforming Employee Experience
Augmentation: Warnings that robots will replace humans have given way to a more nuanced and productive discussion.
Workers in Huntington Ingalls Industries’ shipyard use augmented reality to help build giant complex vessels such as aircraft carriers and submarines. They can “see” where to route wires or pipes or what is behind a wall before they start drilling into it.
Future-readying: providing employees with the skills they need to keep up with the pace of change. In the past few years, this has given rise to new models of managing learning and development in organizations, led by a new kind of chief learning officer, whom we call the transformer CLO
Flexforcing: To respond to fast-paced digital opportunities and threats, companies also need to build agility into their talent sourcing systems. As automation and AI applications take over tasks once performed by humans, some companies are multiskilling employees to make the organization more agile.
Transforming Business Models
three elements supporting business model transformation: digital enhancements, information-based service extensions, and multisided platforms.
Although brain-computer interfaces (BCIs) are the heart of neurotech, it is more broadly defined as technology able to collect, interpret, infer or modify information generated by any part of the nervous system.
There are different types of it—some is invasive, some isn’t. Invasive brain-computer interfaces involve placing microelectrodes or other kinds of neurotech materials directly onto the brain or even embedding them into the neural tissue. The idea is to directly sense or modulate neural activity.
Noninvasive neurotech is also used for pain management. Together with Boston Scientific, IBM researchers are applying machine learning, the internet of things, and neurotech to improve chronic pain therapy.
As new, emerging technology, neurotech challenges corporations, researchers and individuals to reaffirm our commitment to responsible innovation. It’s essential to enforce guardrails so that they lead to beneficial long-term outcomes—on company, national and international levels. We need to ensure that researchers and manufacturers of neurotech as well as policymakers and consumers approach it responsibly and ethically.
